I am an artist and academic who makes large scale immersive artworks or ‘other worlds’ engaging natural or scientific phenomena as its starting point and using sound-design, narrative and site as methods for immersion.
The artworks engage directly with abstract phenomena, neural architectures, desirous matter, the edges of our atmosphere, the gravitational pull of the moon on our earth, or the sonic signature of a fathoms depth. Creating sonic and sited atmospheres, these artworks attune and immerse participants in phenomena that we might not ordinarily sense.
As I began the fellowship I wanted to develop and extend my practice by working with augmented reality. I was curious how AR can appear to disconnect us from our physical sensory experience of the world but conversely has the capacity to connect us intimately with other augmented experiences layered on top of, or integrated within our physical world.
My initial research ideas were focused on developing embodied experiences through AR technology:
How can AR create ‘atmospheres’ that exist within the headset and externally, necessary for embodied immersive experiences –what are the cues that allow participants to inhabit other worlds?
Through augmented sonic and situated environments how can we enhance the precognitive sensing that takes place before the engagement of the intellect to form immersive experiences?
How can sound and site create participatory experiences for immersion that are collective or shared forming a ‘thickening’ or deepening of this shared experience?
How does narrative play a part in sonic and sited AR environments to create a form deep ‘attunement’ necessary for immersion into scientific phenomena that negotiates a pathway between the intellect and the sensory?
Can AR extend co-participation?
I was very lucky to work with academic fellow Coral Manton at Bath Spa University, and Jay Auborn at dBs Bristol. I managed to get a Magic Leap from the States and we began a series of experiments, expanded the team including Phillip Liford and Georgio Cortiana both from dBs Bristol.
To start experimenting with the headset, I created a new artwork Between Us using narrative, sound and moving image.
Illustration by Skye Liu Tianzi 2018
Between Us is an Augmented Reality audio/visual artwork about desire, solar physics and the impossibility of touch. Between Us is made for Magic Leap and Sennheiser AMBEO 3D audio.
Drawing on quantum physicist Karen Barad’s work on touch, the narrative in Between Us is at once seductive and unsettling.
The participant inhabits an audio/visual world where they are addressed directly through the narrative. Immersed in the 3D soundscape and visuals, the participant is both audience and artwork, guided by a disembodied voice that speaks of solar physics and the haunting nature of desire, the anticipative action of touch.
We experimented with the technology, testing the differing aspects of the headset. One of the main technological aims was to get the Magic Leap headset to ‘talk’ to other devises, initially a Subpac which uses vibration to transmit sound through the body. We used Open Sound Control (OSC) and wrote scripts to run on the Magic Leap which then sent messages to a laptop over a network throughout the test. This then communicated with Max/MSP sending signals to the Subpac on certain parts of the narrative, causing sub frequency soundwaves to resonate with the users body. Using different modulators with Max/MSP, we created a dynamic haptic feedback response for the viewer.
Between Us allows us to sonically and visually inhabit the temporary intimate fusing of the outer edge of our atmosphere as it meets the sun’s radiative action – the ionosphere where the sun and the earth first touch. Our skin is constantly filtering air, reacting to temperature changes, signaling hormonal drives, fears and desires. In likening the ionosphere to the skin of our bodies we may by able to imagine the invisible interactions taking place between things and selves, things and things, and selves and selves. In sonifying this interaction we inhabit where the sun and the earth meet, where the ionosphere draws towards the sun like the pilomotor reflexes of the tiny hairs on our skin. The desirous nature of Between Us evokes the dynamic interplay between two bodies or systems, where we inhabit the in-between, the invisible interactions that take place where attraction and repulsion reside. Between Us was premiered at ISEA Lux Aeterna (International Symposium of Electronic Art) June, 2019 in Korea. It was also exhibited at Watershed, Bristol, June 2019 as part of the SWCTN prototype showcase.
We have also began creating another larger artwork, Haunted Geologies, based on Nils Bubandt’s research and writing.
The conceptual ideas framing Haunted Geologies develop long-term concerns in my artistic practice with the manifestation through emerging technologies of entities or fellow beings beyond sensory experience. For this work I draw on two compelling critiques of the anthropogenic age from the Iranian philosopher Reza Negarestani and Danish anthropologist Nils Bubandt, which suggest that spirits locked into geological layers are actors in the current environmental crisis. Bubandt writes of the ‘spectral moment’ in which ‘the deep time of geology becomes the political history of the present’ (2017).
We believe that Haunted Geologies will be one of the world’s first immersive artworks for AR in dome environments. We are developing a pioneering technical innovation that integrates the personalised holographic visuals and sounds of Magic Leap AR with the immersive dome projection allowing a seamless interactive narrative and a truly participatory immersive experience. The innovation developed from the research experiments with Magic Leap AR with the external dome environment began during my fellowship with (SWCTN). I was also part of the ACE delegation to visit SAT Société des arts technologiques in June 2019, enabled me to experience and discuss artworks for dome environments.
Presently we are working with the technologies and securing funding to build this artwork.
Funded by SWCTN and University of Plymouth.