Introduction
Custom-made VR environments can be incredibly valuable for research, but as we discuss in Chapter 6 they can be time-consuming, complex and expensive to create. While the previous chapters have demonstrated that research with VR can be undertaken effectively using pre-existing materials, 360° photos and video can be a striking and straightforward way to get started with the process of creating original immersive experiences. These materials combine 2D images from cameras with two or more wide-angle lenses to create a photo sphere with the viewer at its centre (Figure 5.1). As a tool, it has become popular for virtual field tours (Kenna and Potter, 2018), journalism (Jones, 2017) and tourism (Wagler and Hanus, 2018), giving users the capacity to explore a distanced location in a more naturalistic manner than conventional monodirectional images. Unlike VR environments built with games engines, it is not possible to navigate around the virtual space because 360° environments are merely projections of flat images. Despite this, 360° photography can provide immersive experiences and a high degree of realism with fairly low effort and minimal cost.
360° photograph of woodland in Rotterdam seen in a spherical projection
Source: Tess Osborne360° photograph of woodland in Rotterdam seen in a spherical projection
Source: Tess Osborne360° photograph of woodland in Rotterdam seen in a spherical projection
Source: Tess OsborneThe accessibility of 360° imagery, combined with easy creation, has resulted in an extensive collection of VR experiences. Websites, including YouTube, have produced 360° video content, as well as application developers creating games and other experiences. In this chapter, we reflect upon the strengths of 360° photography for experiencing real-world spaces in VR and the work done on therapeutic landscapes. We then turn to discuss the sensory qualities of VR as an ocularcentric experience. Finally, we reflect upon a pilot we undertook where participants were exposed to urban and natural stimuli through audio and 360° imagery.
Travelling through 360°
One of the main benefits of using 360° imagery, beyond the simplicity and easy access, is that it can provide a more realistic-looking virtual environment than those discussed in earlier chapters. 360° imagery is a visual copy of the real world rather than a computer-generated replication. It is, therefore, no surprise that 360° imagery has become increasingly popular in the tourism sector as a promotional and marketing tool (Guttentag, 2010), where it acts as a ‘try-before-you-buy’ option for potential tourists. Thus, 360° imagery has been a popular addition to the promotional activities of national tourist boards, hotels and travel agencies in recent years. For example, Visit Scotland developed a multimedia smartphone app called ScotlandVR,1 which included animated maps, photos and 360° video to allow people to visit 26 attractions without leaving their homes (Gibson and O’Rawe, 2017). The app was seen as an excellent example of how VR is ‘far from being a fad or gimmick, [but] revolutionising the way people choose
With this realistic quality in mind, 360° imagery also has the potential to enhance accessibility. Just as with the story of Val and Ian in Chapter 3, it can be used to immerse people in spaces that would be difficult, or even impossible, for them to access. Discover Cracow (2016), for example, produced a three-minute-long promotional 360° video called ‘Auschwitz-Birkenau Walkthrough’, which depicts the concentration camp’s interior and exterior spaces. It is a powerful piece of media, combining emotive music with both mobile and stationary video footage; it is no surprise that people were leaving comments along the lines of, ‘Even though I’m not there I can feel it,’ (Discover Cracow, 2016) on the video’s YouTube page. We used this 360° video in a methods workshop in Washington DC to allow participants to experience the various spaces of the camp and how these were subsequently reflected in the architectural design of the city’s United States Holocaust Memorial Museum (Osborne and Jones, 2021). Many of the workshop attendees had only ever seen Auschwitz-Birkenau through 2D photographs and videos. Yet, immersion in the 360° video helped the attendees feel that they had a presence in the environment, as if they were actually visiting the site. This transcendent quality of 360° video makes it an effective tool for researchers: it can create a greater connection to a location compared with 2D imagery and prompt more nuanced reflections from users.
360° therapeutic landscapes
Aside from the fantastic work undertaken in tourism studies, 360° imagery has been used to great effect in research around therapeutic landscapes. The term ‘therapeutic landscapes’ was first coined by Wilbert Gesler in 1992 to explore why certain places seem to have healing qualities, such as green and blue spaces, spas and religious spaces. Since then, the concept and its applications have evolved and expanded with new foci, spaces, methods, and approaches being evaluated, including digital and virtual places (Bell et al, 2018). It is frequently argued that green and blue spaces have a restorative effect (Bell et al, 2018), however, some groups of people, such as poorer communities and the less mobile, may have worse or limited access to these spaces. Therefore, it is no surprise that a growing body of research has focused on understanding the
This body of work has shown how being immersed in 360° nature videos can lead to a therapeutic effect and improved mood (White et al, 2018). Browning et al (2020), for example, studied exposure on mood and restorativeness in three settings: (a) an outdoor forest setting, (b) an indoor setting with no visual or auditory stimulation, and (c) a 360° video of the same forest with noise-cancelling headphones. Using skin conductance and self-reported survey measures, Browning et al were able to show how nature exposure outdoors boosted a positive mood, while VR preserved a good mood compared with sitting indoors with no nature exposure, which diminished the person’s mood. The study shows the promise of 360° imagery for mental-health promotion, which is coupled with its easy accessibility and affordability, and how research with VR can be applied in and outside a lab setting.
Although this book focuses on the methodological approaches of VR, the work of these various scholars and charitable foundations, such as Virtual Dream, demonstrate the ways in which VR can provide relief and encourage well-being. With the affordability of using 360° imagery, easy distribution, and accessibility, it is no surprise that many therapeutic experiences have been developed – especially during the COVID-19 pandemic. Covid Feel Good, for example, is a weekly social self-help virtual therapeutic experience using a 360° video of a ‘Secret Garden’. Riva et al (2020) were able to demonstrate that repeated ten-minute immersions in the Secret Garden led to a statistically significant reduction in anxiety, depression and perceived stress, as measured by the Depression Anxiety Stress Scale and Perceived Stress Scale. This therapeutic-focused work demonstrates the methodological possibilities of 360° imagery
Sensory VR
Notwithstanding the excellent research possibilities arising from the accessibility of 360° photography, it is crucial to consider the whole range of sensory stimuli. We have already shown that there can be sensory mismatches when in VR, such as cybersickness occurring where the visual and the haptic experiences do not align (see Chapter 3). 360° imagery, by its very nature, is a visual experience. Yet, touch, smell, hearing and sight combined are all relevant to how we understand our surroundings. The multisensory has been shown to play a crucial role in our emotional and embodied experiences of place (Rodaway, 2002). Thus, it is fair to assume that VR immersion can be significantly enhanced by engaging senses beyond the visual (Dinh et al, 1999).
Audio is easily incorporated into the VR experience using headphones or located speakers. Hearing is one of the key senses for creating immersive experiences and directly contributes to the sense of presence and therapeutic responses discussed. Annerstedt et al (2013), for example, found that stress recovery can be advanced by adding sounds of nature to a virtual green environment in a laboratory setting. We will elaborate on audiovisual stimuli in the case study later in this chapter. It is important to stress that the audio used in a 360° video can significantly enhance emotional responses. Indeed, the emotive music overlayed on the Auschwitz-Birkenau video reinforces the eeriness and numbness that can occur when watching the video. Thus, using an overlaid audio track can be a handy tool for developing those emotional connections while immersed.
While using an audio track may stimulate an emotional connection, it may also create a disconnect in relation to presence. While the user may be seeing Auschwitz-Birkenau,
Incorporating audio is essential, and is also relatively easy to do compared with the other major senses, namely smell and touch. These are much more challenging to simulate, yet also very important to create a well-rounded and fully immersive experience. In a visit to the Human Interface Technologies Team at the University of Birmingham in 2018, Phil and Tess spoke with Bob Stone and his research group about the various ways they create realistic immersion in the VR environments they create for military training and healthcare. The set-up in their facility was exceedingly impressive, with all the latest hardware and a reproduction of the interior of a
There are both high- and low-tech ways to employ smell when working with HMDs beyond simply filling a whole room with an odour. The FeelReal Sensory Mask, as an example of a high-tech solution, is a newly developed multisensory mask that can stimulate water mist, wind, heat, vibration and over 250 different aromas. It is certainly an impressive piece of kit with its own desktop app for adding scent to video files. Still, it is an extra cost on top of the HMD. Additionally, each bespoke aroma set (such as one for Arizona Sunshine) costs $50, and learning is needed on how to operate a new piece of software. Based on our experience of artificial aromas in Bob Stone’s lab, it is hard to gauge how realistic these artificial smells are. Nonetheless, it is promising that these technologies are being developed and can add new nuances to VR research. We look forward to seeing the sensory masks make progress in the future and possibly become a staple in VR research.
Until then, and for those who prefer to work with a low-tech solution, a sensory tray is a simple but very effective solution. The Living Environments for Healthy Ageing project used a combination of 360° imagery and multisensory simulation experiences, using a sensory tray to explore the benefits of bringing natural environments into the physical space of residential care homes (Scarles et al, 2020). Rather than using artificial smells exclusively, the team created trays filled
Case study: mismatched sensory stimuli
The salutogenic effects of exposure to natural spaces to promote well-being is well established (Bell et al, 2018). Still, such spaces are unevenly distributed, with poorer communities and those who are less mobile having worse access. Exposure to 360° imagery can reproduce some of the sense of being in these spaces, but as we have reflected in this chapter, immersion in a virtual environment requires more than the visual alone, regardless of how realistic that visual environment is. The focus of this study was not to immerse people in a realistic environment, but to create a scenario where the visual stimuli mismatched with the auditory stimuli to explore which sensory stimuli (visual or audio) had the strongest effects on psychological and physiological response. Eleanor, who led this study, adopted a mixed methods approach with pre- and post-VR immersion questionnaires, using the Profile of Mood States (POMS) test alongside biosensing measures, closing the study with a brief semi-structured interview.
Creating the VR environments
Although there is an abundance of 360° imagery available online, we opted to create our own materials to ensure that
Walking to hide behind a tree in Cannon Hill Park while recording a 360° video, seen in an equirectangular projection
Source: Phil Jones and Tess OsborneWalking to hide behind a tree in Cannon Hill Park while recording a 360° video, seen in an equirectangular projection
Source: Phil Jones and Tess OsborneWalking to hide behind a tree in Cannon Hill Park while recording a 360° video, seen in an equirectangular projection
Source: Phil Jones and Tess OsborneFollowing successful 360° imagery capture, the video and audio files were edited and combined using the Gear 360 Action Director software. Two five-minute videos were created that deliberately mismatched the audio and the video: an urban scene with green-space audio and a natural scene with the sounds of traffic noise. It is important to stress that more editing can be done to the footage should it fit with the study. For example, Zulkiewicz et al (2020) added visual effects to their 360° video to portray the sensory experience of a migraine. Indeed, just as with a 2D photograph or video, it is possible to tailor the experience to your needs, but, due to
The study
In this study, we immersed the participants in mismatched sensory scenarios using a stand-alone HMD with three degrees of movement (Oculus Go) and headphones (Bose QuietComfort 25 Acoustic Noise Cancelling). The participants were immersed for five minutes in each scenario while seated in the researcher’s house in Birmingham. Sitting down meant that participants were not exposed to most of the risks linked to trip hazards and the anxiety that can be associated with ambulatory VR (Coldham and Cook, 2017). This also meant that we could avoid bringing signal ‘noise’ from physical movement into the biosensing element of the project (see Osborne and Jones, 2017).
Unlike the other case studies in this book, this took more of a quantitative approach to the experience, using biosensing and the POMS surveys as the primary methods. The POMS test is considered the standard method for studies concerned with assessing participants’ subjective moods and involves asking the participants to indicate on a five-point scale how much they agree with the sentence, ‘I am currently feeling …’ and then a list of 40 moods. Biosensing, on the other hand, measures the automatic, or unconscious, physiological responses that can indicate emotions or stress response. For example, an increase in perspiration combined with a decline in skin temperature (a cold sweat) would indicate stress (Osborne and Jones, 2017).
The pilot study had a limited number of participants (three, all female, aged between 22 and 24) and unfortunately the larger planned campaign of data collection was not completed because of the interruption caused by the UK’s first COVID-19 lockdown in March 2020. Still, the pilot data indicates that people expressed greater feelings of confusion, anger
Despite its limitations, this pilot does demonstrate some of the methodological opportunities that come from working with 360° imagery. The ease of capturing these virtual environments and editing the footage allows the researcher to control the environment in ways that would not be possible in the real world, whether that is changing the sensory stimulation or visual tweaks such as removing litter from the images. Despite the benefits that this control gives the researcher, there is still some debate about the usefulness of VR within research into salutogenic environments. This is mainly due to uncertainty about the extent to which virtual environments can act as substitutes for real environments, with certain subtleties being lost (such as the movement of water in still photos of blue spaces – see Gao et al, 2019). Nonetheless, the adaptability of 360° photography creates an opportunity to explore different environments with participants. This gives us better understandings of people-environment relationships and of how we can modify real spaces to maximise beneficial well-being effects.
Conclusion
In this chapter, we have explored the challenges and opportunities presented when working with 360° photography, which
There is an abundance of 360° content online that can be reused to expand the locations and topics considered in our research projects, and which can also be an object of study in itself. Currently, the majority of research on 360° content has been undertaken in tourism studies, meaning that there are many opportunities to undertake analyses and projects from different disciplinary perspectives. Methods such as content analysis and participatory work can be employed with this material, benefiting from VR’s unique properties around embodiment and immersion.
Despite all these opportunities, the true strength of 360° photography lies in its ability to easily create virtual environments that can be tailored to whatever the research project needs, whether that is mixing up the audiovisuals (as in our case study) or editing the footage for creative or artistic effect (such as Iñárritu, 2017 and Zulkiewicz et al, 2020). Additionally, since 360° photography is a visual copy of the real world, it is more likely to induce a state of presence – the feeling that you are somewhere else. While this may feed into the ethical issues discussed in Chapter 3, if you think along the lines of spatial manipulation, it demonstrates the versatility
A key point to note is that, while 360° photography may give a strong sense of presence, it still differs substantially from the real world. A big factor in this disconnect between the real and virtual is the privileging of vision over the wider senses. While sight may be our dominant sense as a species, we use smell, sound and touch to engage with the environment around us, whether in the real world or VR. The audio element is more straightforward to incorporate than smell and touch, but we have shown that there are various innovative ways to incorporate the olfactory and haptic, such as scent pots, masks and sensory trays. The case study demonstrates one of the ways researchers can explore the senses in VR, but there are many other opportunities to use 360° imagery to explore embodied experiences, using either pre-existing materials or creating our own.
References
Annerstedt M, Jönsson P, Wallergård M, Johansson G, Karlson B, Grahn P, Hansen AS and Währborg P (2013) Inducing physiological stress recovery with sounds of nature in a virtual reality forest – results from a pilot study. Physiology & Behavior 118: 240–250.
Bell SL, Foley R, Houghton F, Maddrell A and Williams, AM (2018) From therapeutic landscapes to healthy spaces, places and practices: a scoping review. Social Science & Medicine 196: 123–130.
Browning MH, Mimnaugh KJ, van Riper CJ, Laurent HK and LaValle SM (2020) Can simulated nature support mental health? Comparing short, single-doses of 360-degree nature videos in virtual reality with the outdoors. Frontiers in Psychology 10: 2667.
Coldham G and Cook DM (2017) VR usability from elderly cohorts: preparatory challenges in overcoming technology rejection. 2017 National Information Technology Conference (NITC). Colombo, OH, pp 131–135.
Dinh HQ, Walker N, Hodges LF, Song C and Kobayashi A (1999) Evaluating the importance of multi-sensory input on memory and the sense of presence in virtual environments. Proceedings IEEE Virtual Reality: 222–228.
Discover Cracow (2016) Auschwitz-Birkenau walkthrough – 360° movie. High quality 4K, available at: www.youtube.com/watch?v=a90WJJVLjZQ [Accessed 22 July 2021].
Gao T, Zhang T, Zhu L, Gao Y and Qiu L (2019) Exploring psychophysiological restoration and individual preference in the different environments based on virtual reality. International Journal of Environmental Research and Public Health 16(17): 3102.
Gibson A and O’Rawe M (2018) Virtual reality as a travel promotional tool: insights from a consumer travel fair. In Jung, T and Tom Dieck M (eds) Augmented Reality and Virtual Reality: Empowering Human, Place and Business. Cham: Springer, pp 93–107.
Guttentag DA (2010) Virtual reality: applications and implications for tourism. Tourism Management 31(5): 637–651.
Hughes CJ and Montagud M (2020) Accessibility in 360° video players. Multimedia Tools and Applications. doi.org/10.1007/s11042-020-10088-0
Iñárritu, AG (2017) Carne y Arena, Fondazione Prada and Legendary Entertainment.
Jones S (2017) Disrupting the narrative: immersive journalism in virtual reality, Journal of Media Practice 18(2–3): 171–185.
Kenna JL and Potter S (2018) Experiencing the world from inside the classroom: using virtual field trips to enhance social studies instruction. The Social Studies 109(5): 265–275.
Mathysen D and Glorieux I (2021) Integrating virtual reality in qualitative research methods: making a case for the VR-assisted interview. Methodological Innovations 14(2): 1–7.
Medrano C (2020) Carne y Arena at Stanley Marketplace is an emotional virtual reality experience, available at: https://www.5280.com/2020/11/carne-y-arena-at-stanley-marketplace-is-an-emotional-virtual-reality-experience/ [Accessed 30 July 2021].
Moran D (2019) Back to nature? Attention restoration theory and the restorative effects of nature contact in prison. Health & Place 57: 35–43.
Osborne T (2021) Restorative and afflicting qualities of the micro-space encounter: psychophysiological reactions to the spaces of the city. Annals of the American Association of Geographers.
Osborne T and Jones P (2017) Biosensing and geography: a mixed methods approach. Applied Geography 87: 160–169.
Osborne T and Jones P (2021) Virtual reality and memorials: (re)building and experiencing the past, in J Micieli-Voutsinas and AM Person (eds) Affective Architectures: More-Than-Representational Geographies of Heritage. Abingdon: Routledge, pp 252–266.
Riva G, Bernardelli L, Browning MHEM, Castelnuovo G, Cavedoni S, Chirico A, Cipresso P, de Paula DMB, Di Lernia D, Fernández-Álvarez J, Figueras-Puigderrajols N, Fuji K, Gaggioli A, Gutiérrez-Maldonado J, Hong U, Mancuso V, Mazzeo M, Molinari E, Moretti LF, Ortiz de Gortari AB, Pagnini F, Pedroli E, Repetto C, Sforza F, Stramba-Badiale C, Tuena C, Malighetti C, Villani D and Wiederhold BK (2020). Covid feel good – an easy self-help virtual reality protocol to overcome the psychological burden of coronavirus. Frontiers in Psychiatry 11: 996.
Rodaway P (2002) Sensuous Geographies: Body, Sense and Place. London: Routledge.
Scarles C, van Even S, Kelpacz N, Guillemaut JY and Humbracht M (2020) Bringing the outdoors indoors: immersive experiences of recreation in nature and coastal environments in residential care homes. e-Review of Tourism Research 17(5): 706–721.
Wagler A and Hanus MD (2018) Comparing virtual reality tourism to real-life experience: effects of presence and engagement on attitude and enjoyment. Communication Research Reports 35(5): 456–464.
White MP, Yeo NL, Vassiljev P, Lundstedt R, Wallergård M, Albin M and Lõhmus M (2018) A prescription for ‘nature’ – the potential of using virtual nature in therapeutics. Neuropsychiatric Disease and Treatment 14: 3001–3013.
Zulkiewicz BA, Boudewyns V, Gupta C, Kirschenbaum A and Lewis MA (2020) Using 360-degree video as a research stimulus in digital health studies: lessons learned. JMIR serious games 8(1), e15422.
