Light Years Coast VR & 360 Software Art

LIGHT YEARS COAST VR & 360 SOFTWARE ART

In 2018 Anthony was appointed as an Immersion Fellow for the South West Creative Technology Network . The role involved exploring the possibilities of several forms of immersive media in order to better understand the potential and correct uses of such media. It was study to find out the differences between these different forms, by directly comparing the same project, output through different forms of 360/virtual reality.

To achieve the comparison, Anthony took his ongoing project, Light Years: Coast and updated it (previously not updated since 2014) in order to make three compatible versions, a large projected version, a 360 video version and a full stereo VR version. 

Light Years: Coast is a virtual painting, an semi-interactive exploration of the Dorset coast, featuring paintings by Jeremy Gardiner. It is a project that originally started in 2002, with several incarnations as an immersive, projected experience. As an experience, it involved a 15 minute boat-like trip up and down the coast, as a virtual gallery, seeing landscape paintings in the approximate location of the subjects they featured.

In Light Years: Coast weather data affects the environment, causing tides and waves to change, rain to fall. Additionally sound recordings at further sensory input. All the landscape (gathered from LIDAR data) of the coast is covered in elements of painting, and the sky is also painting too, gradually fading from one sky to another. Light Years: Coast is about representation of time in a landscape, from current time, to geological time, to sped-up time and seasons.

Light Years: Coast is a piece of software art created in a game engine, and it needed some updating and adaptation to work with the VR headset and updates in the development environment it was created in. In theory, converting a 3D experience into a virtual reality experience is straightforward, as VR is just a stereo virtual camera, that takes it position and rotation information from the headset, but in practice, working on a project that was initially programmed 10 years earlier was more complicated than at first appeared. Anthony worked with his assistant Emma Klasse and used an HTC Vive Pro headset to create a virtual reality version.

The 2010-2014 Light Years: Coast experience was a boat trip along the coast and as mentioned, the intention was to convert the project to a VR boat trip. The boat idea was a powerful experience in virtual reality, the motion of the boat as it rode up and down the waves was very physical. It tricked the brain into thinking that it’s rolling back and forth, side to side. Essentially a kind of nausea inducing motion sickness. It should be noted that this isn’t motion sickness as you would experience in a car, where the body feels its travelling but the brain doesn’t. In this case the brain thinks you are moving when the body doesn’t. The effect is similar but different.

Either way, the nausea element of the project was not a desirable outcome. It would have been a distraction in being able to compare the different versions, and not a pleasant experience for a general public audience. Thus began the work to change the form of motion, whilst keeping nausea to a minimum. After some experiments with different types of navigating the space, a very slow acceleration and moving in slight arcs was settled upon. The viewer/participant would travel around from one featured painting (by Jeremy Gardiner) to the next. Slowly speeding up and then slowly decelerating. On testing, most people did not find this nauseating. To further update the project we took photographs of new paintings by Jeremy, and positioned them in specific locations around the coast, just west of Swanage, covering an approximately 3 mile stretch of coast.

The team started to experimenting with different settings how the virtual stereo camera behaved (which represents the user/viewer), it was realised that a sense of scale was more noticeable in VR than just looking at the screen. Whilst wearing a VR headset, the user ‘owns’ their scale, and this is dependent on the separation distance of the two virtual cameras that VR uses, which would usually match the separation distance between the left and right eye. However, making the virtual separation distance large (much larger than life), made the user feel like a giant, standing in the sea, up to one’s neck. It was a bit suffocating. It also had the effect of making the landscape feel like a model. It was not a feeling that was possible by just looking at the screen version, where your sense of scale is more imagined. Increasing the camera separation also implied adjusting the angles of the cameras, pointing them in towards each other, which has the effect of increasing the sense of 3D depth. This could also add a sense of nausea, particularly as one passed through the painting. Ultimately, these were interesting experiments but not the desired outcome. The aim was to try and give the person an accurate sense of scale to the coastal scape, and thus making the paintings in the environment appear super-sized.

Getting the virtual reality version correct led automatically to the projection version being correct. The only difference was that the projection version uses one camera (it’s a mono experience), and it didn’t include movement interactivity. This made it feel more like a film, although it wasn’t a recorded experience.

To create the 360 degree version that would be viewable online via YouTube, it was a case of screen grabbing the whole experience in equirectangular format, i.e. screen capturing all 360 degrees, in round, but mapping that to a rectangle image (as per the example above), at 8k resolution.  This involved thousands of screen grabs running overnight on the computer, then joined together to create one 360 movie. When viewed on a 360 platform, the user can look around in the space, but, as a recorded experience. 

Thus 3 versions of the new Light Years: Coast were created:

1. A mono camera version for projection or computer screen, with real-time graphics.
2. A mono camera, 360 recorded video version, for YouTube.
3. A stereo camera, real-time graphics version for virtual reality