ONE: What is VR and why use it in research?

This introductory chapter explores the research potential offered by working with VR technology and seeks to demystify VR for non-specialist scholars. Definitions of VR, MR, XR and AR are explored, noting that the focus of the book is on VR as it appears within head-mounted displays. We move on to consider the reasons why researchers may wish to employ VR within their projects, focusing on two main areas: VR as an object of study, and VR as a methodological tool. We also emphasise the significance of VR in generating a strong sense of immersion and presence within virtual environments. Contemporary VR research is situated within a brief history of how the technology has evolved. Finally, we give an overview of the chapters in the remainder of the book.

Introduction

The appeal of popular entertainment is to let us escape our mundane everyday realities and slip into another world filled with excitement and adventure. From TV and films to novels and games, different forms of media allow us to picture a different life. One of the reasons why the idea of VR appeals so much is because it goes beyond merely imagining ourselves in a different world and allows us to actually be there, living a different life in a digital realm. The classic William Gibson novel Neuromancer, TV shows like Caprica and many movies, from Lawnmower Man and the Matrix to Ready Player One, have created fictional futures where we can plug our consciousness into digital platforms and be transported into a virtual world.

Sadly, however, the reality has never quite managed to live up to the fiction. VR is a clunky, imperfect experience. Head-mounted devices (HMDs) for viewing VR material, along with associated technologies for monitoring bodily movements and reproducing them in virtual environments, can be expensive and awkward to use. Despite decades of hype, consumers have not rushed out in huge numbers to buy the equipment required for immersive VR experiences, which remain of relatively niche interest. Nonetheless, these technologies can offer incredibly compelling experiences, where you really do believe that you are being attacked by zombies, flying a plane or wandering around a fantasy kingdom. It is these qualities of believability and immersion that means VR offers some very exciting research opportunities for social scientists and humanities scholars, even those who have little interest in the technical details of how it works.

This short book is intended to serve as an introduction to using VR within research projects. There can be a perception that work using VR requires a great deal of technical expertise; indeed, the majority of projects in this area to date see researchers coding their own customised virtual experiences. This does not, however, need to be the case. We examine a variety of possible methodological approaches to using VR, building in complexity as we go through the book. Each of Chapters 2 to 6 undertakes a critical review of approaches in different methodological areas and includes a worked example taken from our projects within the University of Birmingham’s Playful Methods Lab. The lab serves as a base for postgraduate and postdoctoral research examining different ways to integrate new technologies into qualitative social science projects.

This introductory chapter explores the research potential offered by working with VR technology and seeks to demystify VR for non-specialist scholars. Following brief definitions of terms, we explore the reasons why researchers may wish to employ VR within their projects before we go on to examine questions of immersion and presence. We then situate current VR research within a brief history of how the technology has evolved. Finally, we give an overview of the chapters in the remainder of the book.

Defining terms

Our everyday activities combine sensory inputs from both the material and virtual worlds. From a Zoom call with colleagues to finding a restaurant using a smartphone, much of this material-virtual crossover is now so routine as to be unremarkable. VR and related technologies scale up this blending of the physical and digital by creating whole virtual environments that we can interact with. One of the problems with working in this field, however, is a dense thicket of terminology that can be somewhat off-putting for all but the most technically minded. Virtual reality (VR), augmented reality (AR), mixed reality (MR) and extended reality (XR) are commonly discussed, sometimes with overlapping or even contradictory definitions. We outline these briefly here to help clarify our focus within this book.

XR can be thought of as an umbrella term for different technologies that blend together the virtual and the material worlds to different degrees. According to Milgram and Kishino’s (1994) original definition of MR we can therefore consider MR and XR to be essentially equivalent. They argued that MR represents a ‘virtuality continuum’ (Figure 1.1) that exists between completely material and completely virtual environments. Indeed, Microsoft use ‘Mixed Reality’ as a brand name for a number of products that blend the virtual and material in different ways.

Figure 1.1:
Figure 1.1:

Milgram and Kishino’s (1994) Virtuality Continuum shows how mixed realities blend material and virtual elements to different degrees

Source: Redrawn by Chantal Jackson

Broadly speaking, AR can be thought of as a range of systems that allow digital objects to appear within material spaces. An example of this technology is in combining images from the camera of a mobile phone with 3D digital models such that, for example, you could see what a particular sofa will look like in your living room (Viyanon et al, 2017). More sophisticated systems use occlusion, which means that a digital object can seem to disappear behind a physical object – for example if a person walks in between the camera and your virtual sofa. Some authors argue that where AR uses occlusion it should be defined as MR (Irvine, 2017) despite this going against the original notion of MR as a wider continuum.

This kind of ambiguity is not terribly helpful in what is already quite a confusing field. We do not even have a clear agreement about how VR itself should be defined. It sits at the end of the spectrum where the virtual world takes primacy over the physical in terms of the dominant experience for users. Even at an early stage, however, there was a discussion as to whether devices like a head-mounted display (HMD) and hand controllers were crucial to experiencing VR (Steuer, 1992). Again, this leaves some ambiguity as to whether looking at a traditional monitor showing a virtual environment, such as a 3D model of a building, or a video game, can be considered to be VR.

To avoid confusion, our primary interest in this book is in the kinds of research projects that can be undertaken where virtual environments are viewed through an HMD and this is what we will be referring to when talking about VR. Most of these environments can also be viewed using a traditional monitor, but, as we explore throughout this book, there is something tangibly different about experiencing these through an HMD, with its sense of immersion, which we believe adds significant value when employed within research projects.

Why undertake research using VR?

The social sciences and humanities rightly pride themselves on using a variety of research approaches and techniques for collecting data and discovering more about the world. VR is a tool with a great deal of potential that could be more widely used by scholars but is hamstrung by perceived technological complexity. The main opportunity it brings is in allowing people to explore experiences beyond the constraints of the physical world that nonetheless feel as though they are really happening. VR can therefore be used both by researchers and their participants to explore a wide range of social and embodied experiences in both realistic and fantastic environments.

Throughout this book, we demonstrate the two overlapping ways that social scientists and humanities scholars can engage with VR and the richness it can bring to our work: VR as an object of research, and VR as a methodological tool. As a research object, VR is an emerging experiential medium that warrants critical examination. As a methodological tool, when we put on an HMD, we are seemingly transported into environments and scenarios limited only by the imagination and skill of the content creator. The sense of being present in a VR environment shapes emotions and physical responses and can transform social interactions. This quality alone offers an exciting range of potential research opportunities to scholars.

Conducting research outside a controlled lab setting can be messy, with plenty of ‘noise’ and unplanned distractions. VR is a wonderful research tool to mitigate the untidiness of real situations because it allows researchers to work with immersive environments and scenarios in a highly controllable manner. We can create, or replicate, inaccessible or unavailable spaces (such as mountain peaks or restricted heritage sites) and we can work in different virtual locations without the unwanted disruptions that can occur in the real environment (such as the sound of traffic in a park). This engineering of virtual situations and spaces not only extends experience but allows interesting phenomena to be isolated and subjected to rigorous examination.

A great deal of VR research focuses on its technical performance and industrial applications. It is not our intention to examine this work here, although Jung et al (2020) have produced a useful collection that examines cutting-edge projects of this kind. What this does indicate, however, is that working with VR opens many doors for collaboration. VR has been extensively employed within many disciplines and sectors, from computer science to medicine, psychology to the military, as well as in a range of educational and industrial applications. This means that there are exciting opportunities to collaboratively create, test and implement VR projects developed in other disciplines, while building in theories, practices and constructs from the wider social sciences and humanities that could add real value to research.

Looking at projects where VR is used as a methodological tool, these tend to fall into two broad camps, being primarily interested either with effects on the user’s embodiment in VR, or, more generally, on user interactions with the virtual environment – although, of course, both types overlap. We explore some of the history of VR later in this chapter, but note here that there has been a significant acceleration of research in this area following the commercial release of HMDs from Oculus and HTC in 2016. These ‘third wave’ devices resulted in significant reductions in cost, greater ease of use and a large quantity of newly produced VR content to examine. The shift that this represents offers a very clear research gap for a wider group of social science and humanities scholars to engage with VR beyond the usual suspects working in psychology, archaeology and education.

Decreasing cost is a major potential driver here. Although VR does require some investment in equipment, the new generation of HMDs are of an order of magnitude cheaper than the equipment that was available to researchers in the 1990s and early 2000s. In the past, specialist simulators for motor-racing, aviation and military applications could easily run into the millions of dollars (an early review of flight simulators estimated that the most expensive facilities could cost up to $100 million, Baarspul, 1990). HMDs were very large and required highly specialised computers to run them. Likewise, cave automatic virtual environment (CAVE) systems, where images were projected onto the walls and ceiling of a dedicated room or wraparound screen, were complex and expensive to operate. Not only has the hardware reduced in cost since the 2010s, so the software is no longer in the realm of the hyper specialised, with many more existing VR applications available for scholars to work with and customise to their own needs.

Nonetheless, applied VR research projects did not suddenly begin in 2016. Much of the earlier research work was dominated by medical and psychological experiments, but these studies can be of real interest to social scientists and humanities researchers, not necessarily for the science being explored but because they show how powerful VR can be when working with participants. Reger et al (2011), for example, recruited veterans who were suffering from post-traumatic stress disorder (PTSD), and used VR to deliver exposure therapy. Participants were exposed to simulated convoy and patrol scenarios, with the HMD augmented by a controller shaped like an M4 rifle, along with a vibration plate and simulated smells of burning rubber, bodies and weapons fire. The project was considered successful in producing a drop in self-reported symptoms of PTSD among veterans. Although this is an extreme example, it demonstrates how using VR to immerse participants can allow them to explore a scenario in a way that feels very real while remaining in a safe space.

Clinical research using VR tends to take a trials approach, exploring the difference that using these technologies makes compared with traditional therapies. Again, although the specifics of a study like Henderson et al’s (2007) examination of VR in stroke rehabilitation may be of less interest to non-medical researchers, methodologically the use of a controlled trial is quite interesting. It also highlights the fact that, depending on the type of applications, VR may prove less valuable and does not guarantee improved outcomes over traditional approaches. Thus, in contrast to the apparent success of using VR in treating aversion and PTSD, Henderson et al found only weak and mixed evidence of VR being more effective than conventional therapy in rehabilitation for stroke victims.

A great deal of applied work has been undertaken around VR use in education and skills enhancement. There is an obvious advantage to training for dangerous or expensive scenarios in a controlled virtual environment where there are no serious consequences to participants making mistakes. Again, much of the research investment here has been for high-risk situations, such as surgical training (Gallagher et al, 2005) or military interventions (Lele, 2013). A review by Jensen and Konradsen (2018) concluded that, although the evidence was not overwhelming, there were particular scenarios related to skills acquisition where the use of an HMD could be useful. These included memory, spatial tasks, observation and learning to control emotional response.

Makransky et al (2019) offer a note of warning, however, in reporting on experiments where they gave students simulated lab classes both via a traditional monitor and using an HMD. A combination of self-reporting and recording participant electroencephalogram (EEG) while in the simulations showed that, while the sense of presence in a virtual lab increased when using the HMD, the capacity for learning was actually lowered. This serves as a useful reminder that virtual presence in and of itself does not necessarily bring added value. Again, it depends entirely on what outcomes an applied project is concerned with generating. It is also important to think about who your participants are. As Hayes and Johnson (2019) comment, now that VR simulations are increasingly used for training, so designers need to think about how to build in representations of bodies with diverse gender and ethnicity options. Having a virtual body that looks like your own enhances the sense of connection to the learning experience and increases the likelihood that lessons will be transferred to life outside VR.

Archaeology and the heritage sector have enthusiastically embraced the possibilities of VR as a tool for reconstructing past environments. It is a relatively short step from creating a digital 3D reconstruction of a site to making that virtual environment available to explore in an HMD. Reconstructions within archaeology have a long history, including institutions such as open-air museums, like Stockholm’s Skansen, which can be used for both visitor education and research projects. As Schofield et al (2018) point out in discussing a VR exhibit of a 9th-century Viking camp built by their team, the danger is that users may not be able to tell the difference between those elements grounded in sound historical evidence and more fanciful or dramatic interpretations. Nonetheless, archaeologists use many of the same tools as game designers in producing accurate digital models of existing buildings and landscapes, including laser scanning, photogrammetry and drone-based aerial surveys. Where game designers are unashamed about the degree of invention and imagination they inject into representations of real locations (Jones and Osborne, 2020), the 2006 London Charter for the Computer-Based Visualisation of Cultural Heritage set out methodological principles for the creation and use of digital models within research (López-Menchero Bendicho et al, 2017).

One of the key reasons for wanting to record archaeological sites digitally is because many are fragile and difficult to access. High-quality digital records allow students and researchers to visit sites that would otherwise be off limits. The value of using an HMD here is the sense of presence, such as a project that allowed participants to walk around a photorealistic reconstruction of the 5th-century-BC Etruscan Bettini tomb that few would ever be able to experience directly (Jiménez Fernández-Palacios et al, 2017). It is perhaps little surprise that there have also been a number of archaeological VR projects examining maritime heritage (McCarthy et al, 2019). Some of these have cleverly reused older datasets, such as Secci et al (2019) taking historic photogrammetry surveys to create an immersive experience of diving the wreck of the brig Mercurio, which was sunk during the Battle of Grado in 1812. Costa and Melotti (2012) have even gone so far as to argue that VR heritage assets have created new forms of virtual ‘hyper-tourism’, disconnected from the real locations that have been captured. Custodians of the Newgrange stone-age tomb in Ireland, for example, hold a lottery each year for visitors to witness the winter solstice dawn, the experience of which is considered spiritual by some. Exploring Newgrange in VR is a very different experience, but allows this heritage asset to become disconnected from the constraints of its physical location, meaning that many more people can potentially ‘visit’.

Geography is another discipline interested in the spatial qualities of environments. A great many people working in geography create and use different kinds of 3D environments, from climate and river-flow models to visualisations of complex geographic datasets. Mike Batty (1997; Lin and Batty, 2011) discusses these as being virtual geographic environments. Despite the fact that many researchers within different parts of the discipline create virtual geographic environments, surprisingly few geographers have explored the potential for using more immersive forms of VR within their research projects. An important exception to this is the work of the Serious GeoGames Lab at the University of Hull. Building off geomorphologist Chris Skinner’s (2020) gamified 3D flood simulation, for example, historical geographer Briony McDonagh and literary scholar Stewart Mottram have collaborated to build a VR experience. ‘By the tide of the Humber’ reconstructs 17th-century Hull and river flooding at the time of the poet Andrew Marvell as part of the much wider ‘XR Stories’ initiative that seeks to bring storytelling and new technologies together to boost the creative economy in the Yorkshire and Humber region of the UK.

McDonagh and Mottram’s project is unusual in coming out of the qualitative and humanities side of geography because work using 3D modelling is more commonly employed by physical scientists. In part, this reflects an issue that we explore throughout this book, where the technical nature of creating VR experiences tends to exclude social science and humanities scholars unless they can find a skilled collaborator to work with. Many of the case studies that we explore in the subsequent chapters, however, make use of existing VR resources rather than developing new ones.

A really nice example of this approach is a teaching project undertaken by the geographer Patrick Hagge (2019). Undergraduate students were asked to prepare a guided tour of a global location to share as a presentation with the rest of the class. The twist was that they could give the presentation through Google Earth VR. Drawing on the same database as conventional Google Earth, the VR version presents a highly detailed, stereoscopic rendering of different landscapes and settlements. The student presenter wore an HMD and navigated the virtual site that they were giving a tour of, while the rest of the class watched the output of the headset on a traditional screen. This mode of navigation gave the class a more ‘in-person’ perspective than simply zooming around on Google Earth as normal, though it was not without drawbacks as an approach. Some female students in particular were less keen to engage in this voluntary activity. A common concern was the fear of looking ridiculous while cut off from the class they were standing in front of. Again, this is an important point to consider methodologically – not all participants will be happy to don an HMD in public.

Most of what we have discussed so far has related to computer-generated environments presented in an HMD. One can, however, record 360° photos and videos of real environments that can be viewed in a headset. As a technique, this has relatively low barriers to entry in terms of creating content and experiences for participants to engage with – as we will discuss in more detail in Chapter 5. Some professional film-makers and media organisations have created high-quality 360° content that can be reused within research projects. Thus, news reports can give some sense of being in a site at a particular point in history (Watson, 2017), for example the BBC’s 360° report from the Calais ‘Jungle’ migrant camp in 2015. The journalism scholar Sarah Jones has undertaken some really interesting projects in this area, not least examining whether the claims that this technique generates greater empathy among viewers stand up to critical scrutiny (Jones and Dawkins, 2018b). She has also experimented with questions of immersion in 360° media, which is sometimes claimed to generate less sense of presence because it lacks interactivity. Adding heat and smell stimuli while viewers were watching a documentary film shot in Hong Kong’s Chungking Mansions notably increased the viewers’ sense of being present in the scene (Jones and Dawkins, 2018a). Thus, when considering the use of existing 360° footage in a research project, the powerful effects on sense of presence created by the non-visual senses should be borne in mind.

Immersion and presence

One of the reasons why HMDs are so appealing is the ‘wow’ factor (Heim, 2017). Putting a headset on for the first time and being able simply to turn your head and look around a virtual environment that completely surrounds you is genuinely impressive. If you are wearing a more expensive device, realising that you can physically move in that virtual space – walk around, crouch, see the movement of the hand controllers reproduced in front of you – can be genuinely magical. A common response by novice users is literally to gasp.

Once you get past the initial shock, however, one quickly begins to wonder about the point of VR; essentially, what is one supposed to do in these virtual worlds? This lack of obvious applications proved to be a real problem and, after an initial flurry of excitement around 2016 to 2017, a number of companies subsequently abandoned their VR efforts because of customer disinterest. Inexpensive VR devices such as Samsung Gear VR and Google Daydream have been discontinued, with no rush by their manufacturers to create new versions. Nonetheless, from a research point of view, the capacity of HMDs to generate a sense of being in virtual space is a fascinating quality that creates interesting opportunities for projects.

We should, however, briefly pause here to distinguish between immersion and presence. Immersion is a relatively objective measure, determined by the kinds of technologies being employed in a VR system, both hardware and software, that are intended to generate a sense of being located within a virtual world. Presence is more subjective and dependent on the perception of the individual. Different participants can each feel more or less present in a virtual world even when using technologies with the same immersive potential (Bowman and McMahan, 2007).

There are multiple, competing definitions of presence in VR (for a review, see Schuemie et al, 2001). Lombard and Ditton (1997) give a useful starting point for thinking about this, identifying six different markers of presence:

  • Presence as social richness: the warmth felt when interacting with other people in the virtual environment.

  • Presence as realism: whether the medium appears to accurately reproduce elements of the material world.

  • Presence as transportation: the sense of being there.

  • Presence as immersion: the extent to which the senses are convinced by the virtual medium.

  • Presence as social actor within medium: whether the user responds emotionally to a representation of a person within the environment.

  • Presence as medium as social actor: whether the environment itself can be perceived to be a social actor.

Not all of these elements have the same significance at the same time in different VR scenarios. What is interesting, however, is that the emphasis is not simply on how sophisticated the graphics are – although this can be important – but also on the sense of socialisation and co-presence with other people, both real and computer-generated. This is a theme that we will return to in Chapter 4. It is also clear that not only can VR be highly effective at generating emotional response, but emotional response and sense of presence are mutually reinforcing (Riva et al, 2007).

Early academic work on VR focused on its psychological effects and the capacity to fool our different sensory perceptions into reporting that we are in a different location, even going so far as to claim that, ‘The intent of all this [sensory] input is to sensitize the computer to the user, to turn every movement into a creative tool and means of communication’ (Biocca and Delaney, 1995: 63). Commentary about VR in the 1990s was filled with this kind of utopian language, despite the technology being a long way from being able to deliver on these ideals. Even at that early stage, however, experiments showed the power of immersion, with HMD users shown to be significantly faster at an orientation task than those viewing the same virtual environment via a monitor, because of this much greater sense of presence within the scene (Pausch et al, 1997). Users can even develop a sense of physical connection to virtual limbs as depicted within an HMD – a connection that indicates the malleability of our body image (Yuan and Steed, 2010). This illusion that the body itself exists within the virtual environment has been used in clinical contexts, with burns victims reporting significantly less pain when immersed in VR (Hoffman et al, 2001). This sense of connection to the virtual body can be even more noticeable when reinforced with minor haptic effects – a slight vibration of the hand controller when firing a virtual gun adds a sense of solidity to the experience.

The evolution of VR

Histories of VR explore different precursors, from Plato’s cave to Victorian stereoscopic images and Morton Heilig’s Sensorama (Burdea and Coiffet, 2003). The Sensorama was a fascinating (if slightly Heath Robinson) demonstration device that mixed stereoscopic film with sounds, smells and vibrations to give a convincing illusion of being in a remote physical space. Its patent application emphasises potential uses within training scenarios (Heilig, 1962). By the 1970s, the first HMDs were being produced that were the clear predecessors of the devices we use today. The fundamental design principles have not evolved a great deal, with users strapping a box to the front of their head, containing lenses and a pair of screens projecting stereoscopic images to users’ eyes. As the technology developed during the 1980s, different techniques for tracking head and body movement were added. By 1991 Virtuality had produced a high-end VR games machine for use in video arcades (Delaney, 2014) while Sega announced it would be selling a headset for home use by 1993.

The Sega VR never actually made it to market. Like similar contemporary products, such as Nintendo’s Virtual Boy, these 1990s consumer VR headsets failed in part because they made users ill (Rebenitsch, 2015). To make VR work, the screens inside the headset need to have a very high refresh rate, otherwise they appear to flicker, causing nausea. This problem becomes even worse if there is a lag between the user’s head movements and those on screen because this creates motion sickness. It was simply not possible with the technology available in the early 1990s to deliver a product that could track and refresh quickly enough so as not to create these unwanted effects while still hitting a consumer price point.

Although commercial VR flopped in the 1990s, the decade saw rapid development in the power of computer graphics and new possibilities for creating 3D environments, both for gaming and a range of industrial applications. The use of computer graphics within the film industry revolutionised visual effects, while engineers, architects, planners, the heritage sector and others benefited from the ability to create and manipulate realistic digital models. By the early 2000s, consumer-facing games consoles and PC graphics cards had appeared that could render 3D environments in incredible detail. Within a few years, the mass market for smartphones helped to drive the creation of small displays with very high resolution and refresh rates, which would prove to be perfect for HMDs. The first Microsoft Kinect, released in 2010, showed the mainstream potential for computers to track the movements of objects in the physical world and relate these to virtual environments (Zhang, 2012).

The technological pieces were therefore starting to come together to revisit the idea of consumer VR. In 2012, Palmer Luckey founded Oculus VR and sought crowdfunding to develop the prototype HMD he had been experimenting with for several years (Clark, 2014). One of the prevailing jokes about VR at the time was that it had been the next big thing for about 30 years. Nonetheless, there was so much interest in the possibilities of VR that the Oculus Kickstarter campaign was spectacularly successful, meeting its target many times over. To the surprise of some, social media giant Facebook subsequently bought Oculus in 2014 for over $2 billion. This investment reflected the significant advance that the Oculus technology represented over previous efforts in VR. Facebook’s involvement gave a global platform to push it forward.

Oculus Rift developer kits were made available from 2013 and the finished headset received a commercial release in 2016, the same year that HTC released its technically more sophisticated (and expensive) Vive headset. Other companies have also brought headsets to market with varying degrees of success, some using Microsoft’s Mixed Reality platform which built VR support into the 2017 update of Windows 10. These more recent innovations have been characterised by some as the ‘third wave’ of VR (Heim, 2017).

VR systems from the 1970s to 1990s tried a variety of different mechanisms for fooling the senses to convince the user they were present in the virtual world. Data gloves and even full body suits were developed to allow multiple sensory inputs to be synchronised with a virtual experience. In the third wave of VR, however, these approaches have been simplified into combining hand controllers alongside the HMD as the primary mechanisms for interacting with the virtual environment.

The different platforms have applied different technical solutions to tracking bodily movement. First-generation Oculus and HTC headsets used external beacons, which had to be arranged around the space in which the HMD was being used in order to track the user’s movement. While these allowed for accurate tracking up to the scale of a medium-sized room, they were also a nightmare of trailing wires and worked best when there were no other objects in the room (Figure 1.2).

Figure 1.2:
Figure 1.2:

VR demonstration set up for public display. Note the tracking beacons mounted on tripods at the edges of the game area creating a trip hazard of trailing wires, here fenced off with a collection of stools

Source: Phil Jones

Other HMDs use inside-out tracking, which is not reliant on external beacons. Cameras built into the headset work out how the user is moving around a room. This is a less accurate solution for room-scale movement, but much easier to set up and so can be quite a good compromise for research projects undertaken by non-specialists. Most VR products now use two handheld controllers that can be tracked in space either through the HMD’s cameras or external beacons. Older, budget devices used a single ‘pointer’ with basic movements tracked using an internal gyroscope. Newer technology now makes it possible to track hand movements without the need to hold a controller, just using cameras on the HMD. This means that gestures can control different functions within the virtual world in a more naturalistic manner.

There is, however, no getting away from the fact that VR can be temperamental and fiddly to get working. Oculus, HTC and Microsoft all use different software platforms, which are not interoperable. One can buy VR games through Steam – the dominant online marketplace for PC games – but this adds yet another layer of complexity. To give an example of this problem, within the Playful Methods Lab we have done some work using a gaming steering wheel and pedals for driving within VR. Although a very compelling experience when it all works, adding another device brings even more problems to a VR set-up and it can be difficult to persuade all the different components to communicate with each other without a good deal of time spent tweaking, adjusting and resetting. The impression is often of a technology that is really only one step up from the prototype stage.

Beyond the different manufacturers involved, there are different types of HMD that require some consideration when designing a research project, as they offer different costs and benefits depending on what you are trying to achieve. The most powerful set-ups are generally tethered by wire to a computer with a high-specification graphics card. These give the most complete VR experiences, but are also expensive since you need both a headset and a powerful gaming PC. There is also great potential to literally trip over the tether connecting the HMD to the computer while moving around.

Stand-alone headsets, conversely, have lower graphical capability since all the computing power is contained within the headset itself – effectively using the same processors and sensors one finds in a smartphone. At the start of the third wave, smartphone makers briefly experimented with VR platforms where phones could be slotted into a ‘dumb’ headset that was essentially just a box to hold over your face with a lens for each eye – Google’s Cardboard and Samsung’s Gear VR were the best known of these, although they never really got much beyond the gimmicky stage. There are also some examples of more hybrid devices. Some HMDs have an optional wireless transmitter that allows a PC-based VR experience without needing a cable to connect to the gaming computer. Others can be used either as a stand-alone device or plugged into a PC for more advanced experiences.

Most modern HMDs offer tracking with six degrees of freedom (Figure 1.3). This means that the device can not only detect head movement (yaw, pitch, roll) but also positional movement of the body in space (forward-backward, up-down, side to side). Older stand-alone HMDs and the now mostly obsolete smartphone-based VR systems could only track with the first three degrees of freedom, allowing users to turn their heads but not walk around in virtual space. Similarly, the hand controllers on these basic devices offered only limited movement tracking, which meant much less capacity to interact physically with the VR environment via hand movements. These technological considerations can shape what is and is not possible in different research projects, depending on the equipment used. That being said, as devices improve these constraints are being overcome.

Figure 1.3:
Figure 1.3:

Basic HMDs only track head movement via pitch, roll and yaw. More sophisticated VR equipment tracks users’ movements across six degrees of freedom by adding strafe, elevation and thrust across the x, y and z axes

Source: Chantal Jackson

Structure of this book

The purpose of this book is to set out the different ways in which VR can be employed within research projects. It is important to emphasise that VR research does not have to be limited to specialists and technical experts, and we have structured the book to reflect this. The remaining chapters progress through approaches to VR research, starting with the most straightforward and moving through increasing levels of complexity. The book thus progresses from different kinds of projects that can be conducted using existing VR content, with the final chapters examining how to develop original materials that can be explored in VR.

Each of the chapters that follow examines a different approach to VR, critically reviewing the kinds of projects that have been undertaken in these areas. Each chapter also contains a case study from project work undertaken within the Playful Methods Lab to explore how these approaches can be implemented.

Chapter 2 examines perhaps the most straightforward approach to operationalise, where the researcher’s own consumption of VR materials becomes the basis for content analysis. Commercial VR content, both games and other experiences, have been subjected to relatively little analysis by critical scholars. There is great potential, therefore, to adapt some of the interpretative tools from other disciplines, particularly game studies, in order to undertake analysis of these materials. Game studies put an emphasis on playing the text, rather than simply examining the story; autoethnography sits at the heart of this approach. This chapter reflects on the advantages and limitations of autoethnographic approaches for examining VR content as interactive texts. As a worked example of this approach, we reflect on our analysis of Half-Life: Alyx (Valve, 2020), the first big-budget (‘triple-A’) franchise game to be released exclusively for VR.

Chapter 3 moves beyond the researcher’s own perspectives to examine how larger groups of participants can be enrolled in VR studies. The focus is on how existing VR materials might be used in projects with participants as a simpler and lower-cost alternative to building original VR content. We explore the ethics of working with human subjects in VR and the problems of cybersickness, as well as reflecting on the predominance of quantitative methods in projects that analyse participant response. The worked example, conversely, presents a qualitative analysis of a project where 33 regular gamers played the zombie shooter Arizona Sunshine (Vertigo Games, 2016). The exercise revealed a powerful affectual connection to the virtual space, creating a considerably more physically and emotionally intense gaming experience than participants were used to, even as experienced players.

As discussed, definitions of presence within VR lean heavily on questions of social engagement. Chapter 4, therefore, explores the opportunities presented when multiple users interact within the same virtual environment, particularly through social VR platforms. Mechanisms for collaboration within virtual space are examined alongside the critical role that avatar design plays in these interactions. While the nature of the HMD is to cut users off from the world around them, it does allow them to form communities within the virtual spaces they visit. We reflect on some of these issues through a case study of VR Church, where worshippers come together for virtual church services within the social VR platform AltspaceVR (Microsoft, 2015). This provides an opportunity to reflect on the challenges of undertaking ethnographic research with communities in VR.

Chapter 5 moves to look at the most basic form of content creation for VR, producing 360° photos and video. Images from two or more cameras with fish-eye lenses are stitched together to make a photo sphere. When viewed through a HMD, users can turn their heads and look around these images as if from a fixed point at the centre of the scene. As a tool, it has become popular for virtual field tours, journalism and tourism, allowing users to explore a site in the round. Existing 360° content like this can be reused within research projects, but it is also relatively cost effective and straightforward for researchers to generate their own materials for use within specific projects. The chapter focuses particularly on therapeutic landscapes and how 360° content can be considered as part of wider sensory stimulation in VR. This is followed by a worked example of a pilot project examining how the well-being effects of exposure to nature might be reproduced and interrogated through 360° video and audio.

Chapter 6 explores more complex forms of VR content creation, by using games engines to program original materials. Rather than covering the specifics of coding, the chapter reflects on different approaches to building original content, including opportunities for collaboration with skilled practitioners. In exploring why researchers may wish to develop original VR materials, the chapter reflects on two overlapping types of projects: those testing specific scenarios with participants, and those exposing users to novel environments. In the book’s final worked example we reflect on a project where we created VR models of two historic landscapes for use in a workshop examining memory and memorialisation.

Chapter 7 concludes the book, examining what the next steps might be for research by social scientists and humanities scholars interested in using VR within projects.

References

  • Baarspul M (1990) A review of flight simulation techniques. Progress in Aerospace Sciences 27(1): 1120.

  • Batty M (1997) Virtual geography. Futures 29(4): 337352.

  • Biocca F and Delaney B (1995) Immersive virtual reality technology, in F Biocca and M Levy (eds) Communication in the Age of Virtual Reality. Hillsdale, NJ: Lawrence Erlbaum Associates, pp 57126.

    • Search Google Scholar
    • Export Citation
  • Bowman D and McMahan R (2007) Virtual reality: how much immersion is enough? Computer 40(7): 3643.

  • Burdea G and Coiffet P (2003) Virtual Reality Technology, Second edition. Hoboken, NJ: John Wiley & Sons.

  • Clark T (2014) How Palmer Luckey created Oculus Rift. Smithsonian Magazine, available at: https://www.smithsonianmag.com/innovation/how-palmer-luckey-created-oculus-rift-180953049/ [Accessed 6 April 2020].

    • Search Google Scholar
    • Export Citation
  • Costa N and Melotti M (2012) Digital media in archaeological areas, virtual reality, authenticity and hyper-tourist gaze. Sociology Mind 2(1): 5360.

    • Search Google Scholar
    • Export Citation
  • Delaney B (2014) Virtual Reality 1.0 – the 90s: The Birth of VR, in the Pages of CyberEdge Journal. Unknown: CyberEdge Information Services.

    • Search Google Scholar
    • Export Citation
  • Gallagher AG, Ritter EM, Champion H, Higgins G, Fried MP, Moses G, Smith CD and Satava RM (2005) Virtual reality simulation for the operating room: proficiency-based training as a paradigm shift in surgical skills training. Annals of Surgery 241(2): 364372.

    • Search Google Scholar
    • Export Citation
  • Hagge P (2019) Student perceptions of virtual reality usage in a university classroom using ‘Google Earth VR’. Association of American Geographers Annual Conference. Washington DC, available at: https://aag.secure-abstracts.com/AAG%20Annual%20Meeting%202019/abstracts-gallery/22117 [Accessed 9 April 2020].

    • Search Google Scholar
    • Export Citation
  • Hayes A and Johnson K (2019) Cultural embodiment in virtual reality education and training: a reflection on representation of diversity, in M Chang, E Popescu, Kinshuk, N-S Chen, M Jemni, R. Huang et al (eds) Foundations and Trends in Smart Learning. Singapore: Springer, pp 9396.

    • Search Google Scholar
    • Export Citation
  • Heilig M (1962) Sensorama Simulator. US Patent US3050870, available at: http://www.freepatentsonline.com/3050870.pdf [Accessed 3 April 2020].

  • Heim MR (2017) Virtual reality wave 3, in J Gackenbach and J Bown (eds) Boundaries of Self and Reality Online. San Diego: Academic Press, pp 261277.

    • Search Google Scholar
    • Export Citation
  • Henderson A, Korner-Bitensky N and Levin M (2007) Virtual reality in stroke rehabilitation: a systematic review of its effectiveness for upper limb motor recovery. Topics in Stroke Rehabilitation 14(2): 5261.

    • Search Google Scholar
    • Export Citation
  • Hoffman HG, Patterson DR, Carrougher GJ and Sharar SR (2001) Effectiveness of virtual reality-based pain control with multiple treatments. The Clinical Journal of Pain 17(3): 229235.

    • Search Google Scholar
    • Export Citation
  • Irvine K (2017) XR: VR, AR, MR – What’s the difference?, available at: https://www.viget.com/articles/xr-vr-ar-mr-whats-the-difference/ [Accessed 3 April 2020].

  • Jensen L and Konradsen F (2018) A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies 23(4): 15151529.

    • Search Google Scholar
    • Export Citation
  • Jiménez Fernández-Palacios B, Morabito D and Remondino F (2017) Access to complex reality-based 3D models using virtual reality solutions. Journal of Cultural Heritage 23: 4048.

    • Search Google Scholar
    • Export Citation
  • Jones PI and Osborne T (2020) Analysing virtual landscapes using postmemory. Social & Cultural Geography 21(2): 186206.

  • Jones S and Dawkins S (2018a) The Sensorama revisited: evaluating the application of multi-sensory input on the sense of presence in 360-degree immersive film in virtual reality, in T Jung and MC tom Dieck (eds) Augmented Reality and Virtual Reality: Empowering Human, Place and Business. Cham: Springer, pp 183197.

    • Search Google Scholar
    • Export Citation
  • Jones S and Dawkins S (2018b) Walking in someone else’s shoes: creating empathy in the practice of immersive film. Media Practice and Education 19(3): 298312.

    • Search Google Scholar
    • Export Citation
  • Jung T, tom Dieck MC and Rauschnabel P (eds) (2020) Augmented Reality and Virtual Reality: Changing Realities in a Dynamic World .Cham: Springer.

    • Search Google Scholar
    • Export Citation
  • Lele A (2013) Virtual reality and its military utility. Journal of Ambient Intelligence and Humanized Computing 4(1): 1726.

  • Lin H and Batty M (2011) Virtual Geographic Environments .Redlands CA: ESRI Press.

  • Lombard M and Ditton T (1997) At the heart of it all: the concept of presence. Journal of Computer-Mediated Communication 3(2).

  • López-Menchero Bendicho VM, Flores Gutiérrez M, Vincent ML and Grande León A (2017) Digital heritage and virtual archaeology: an approach through the framework of international recommendations, in M Ioannides, N Magnenat-Thalmann and G Papagiannakis (eds) Mixed Reality and Gamification for Cultural Heritage. Cham: Springer, pp 326.

    • Search Google Scholar
    • Export Citation
  • Makransky G, Terkildsen TS and Mayer RE (2019) Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction 60: 225236.

    • Search Google Scholar
    • Export Citation
  • McCarthy JK, Benjamin J, Winton T and van Duivenvoorde W (2019) 3D Recording and Interpretation for Maritime Archaeology. Coastal Research Library, vol 31 .Cham: Springer.

    • Search Google Scholar
    • Export Citation
  • Microsoft (2015) AltspaceVR. Seattle WA: Microsoft.

  • Milgram P and Kishino F (1994) A taxonomy of mixed reality visual displays. IEICE Transactions on Information and Systems 77(12): 13211329.

    • Search Google Scholar
    • Export Citation
  • Pausch R, Proffitt D and Williams G (1997) Quantifying immersion in virtual reality. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques . doi.org/10.1145/258734.258744: 1318.

    • Search Google Scholar
    • Export Citation
  • Rebenitsch L (2015) Managing cybersickness in virtual reality. XRDS: Crossroads, The ACM Magazine for Students 22(1): 4651.

  • Reger GM, Holloway KM, Candy C, Rothbaum BO, Difede J, Rizzo AA and Gahm GA (2011) Effectiveness of virtual reality exposure therapy for active duty soldiers in a military mental health clinic. Journal of Traumatic Stress 24(1): 9396.

    • Search Google Scholar
    • Export Citation
  • Riva G, Mantovani F, Capideville CS, Preziosa A, Morganti F, Villani D, Gaggioli A, Botella C and Alcañiz M (2007) Affective interactions using virtual reality: the link between presence and emotions. CyberPsychology & Behavior 10(1): 4556.

    • Search Google Scholar
    • Export Citation
  • Schofield G, Beale G, Beale N, Fell M, Hadley D, Hook J, Murphy D, Richards J and Thresh L (2018) Viking VR: designing a virtual reality experience for a museum, in Proceedings of the 2018 Designing Interactive Systems Conference, Hong Kong: Association for Computing Machinery, pp 805815.

    • Search Google Scholar
    • Export Citation
  • Schuemie MJ, van der Straaten P, Krijn M and van der Mast CA (2001) Research on presence in virtual reality: a survey. CyberPsychology & Behavior 4(2): 183201.

    • Search Google Scholar
    • Export Citation
  • Secci M, Beltrame C, Manfio S and Guerra F (2019) Virtual reality in maritime archaeology legacy data for a virtual diving on the shipwreck of the Mercurio (1812). Journal of Cultural Heritage 40: 169176.

    • Search Google Scholar
    • Export Citation
  • Skinner C (2020) Flash Flood! A SeriousGeoGames activity combining science festivals, video games, and virtual reality with research data for communicating flood risk and geomorphology. Geoscience Communication 3(1): 117.

    • Search Google Scholar
    • Export Citation
  • Steuer J (1992) Defining virtual reality: dimensions determining telepresence. Journal of Communication 42(4): 7393.

  • Valve (2020) Half-Life: Alyx .Bellevue WA: Valve Corporation.

  • Vertigo Games (2016) Arizona Sunshine. Rotterdam: Vertigo Games.

  • Viyanon W, Songsuittipong T, Piyapaisarn P and Sudchid S (2017) AR furniture: integrating augmented reality technology to enhance interior design using marker and markerless tracking, in Proceedings of the 2nd International Conference on Intelligent Information Processing. Bangkok: Association for Computing Machinery, Article 32.

    • Search Google Scholar
    • Export Citation
  • Watson Z (2017) VR for news: the new reality? Oxford: Reuters Institute for the Study of Journalism.

  • Yuan Y and Steed A (2010) Is the rubber hand illusion induced by immersive virtual reality? 2010 IEEE Virtual Reality Conference (VR). doi.org/10.1109/VR.2010.5444807: 95102.

    • Search Google Scholar
    • Export Citation
  • Zhang Z (2012) Microsoft Kinect sensor and its effect. IEEE MultiMedia 19(2): 410.

  • Figure 1.1:

    Milgram and Kishino’s (1994) Virtuality Continuum shows how mixed realities blend material and virtual elements to different degrees

  • Figure 1.2:

    VR demonstration set up for public display. Note the tracking beacons mounted on tripods at the edges of the game area creating a trip hazard of trailing wires, here fenced off with a collection of stools

  • Figure 1.3:

    Basic HMDs only track head movement via pitch, roll and yaw. More sophisticated VR equipment tracks users’ movements across six degrees of freedom by adding strafe, elevation and thrust across the x, y and z axes

Content Metrics

May 2022 onwards Past Year Past 30 Days
Abstract Views 14 0 0
Full Text Views 580 288 15
PDF Downloads 528 223 11

Altmetrics