Virtual reality (VR) is intended to create an immersive fictional environment that is an alternative to, but indistinguishable from, the real world. As a recent article by Stanford academics (James G. Brown, Jeremy N. Bailenson, and Jeffrey Hancock) observed:

“There is already substantial concern about misinformation in our current information ecosystem, from websites to social media to news content, which is composed primarily of text, images, and video. …Given the potency of immersive VR for novel experiences that go far beyond text, image, and video, the potential for manipulative persuasion using misinformation is particularly alarming.”

Misrepresentation in social media vs in VR

The authors identified a number of key differences between misrepresentation in social media (about which there has been an explosion of research in the last 5 years) and misrepresentation in VR (about which there has been very little research so far).

First, VR is difficult to create, especially compared to the ease with which social media misinformation campaigns can be created using ‘copy-pasta’ language As the authors explain:

“Even with today’s push to build the metaverse and the advent of new creation tools, it remains incredibly time-consuming and difficult to build high-quality, compelling content for VR. Unlike a movie or most video games, a VR scene has to be exhaustively reactive, that is, it has to be able to be rendered from any possible distance or angle. To put this in perspective, a film only shows the viewer one angle or distance at a time, and most video games offer players a very limited set of viewpoints. In VR, a person can choose any one of an infinite number of locomotion/gaze combinations as they view a scene. In a pizza parlor, the viewer might decide to crawl on all fours and then turn their head upward to stare at the bottom of a table, and the scene needs to be built in order to support any possible exploration.”

This means that the creation of VR misinformation is more likely to be a deliberate, calculated endeavour by a malicious actor (e.g. state actors) to create a false belief, rather than naive actors who spread misinformation. This means that there may be both less volume of misrepresentation in the VR world and it may be more readily traceable or identifiable as a major campaign by a well resourced actor.

Second, the authors note that “mis-experience in VR does not automatically lead to misperceptions, just as exposure to misinformation in social media or on the internet does not automatically lead to misperceptions.” Somewhat surprisingly, research has found that people exposed to misinformation on social media about outcome of the last US election are only 4% more likely to believe the election was rigged. Yet there is also no doubting that when some users are gripped by social media misinformation, particularly the vulnerable who are targeted by misinformation campaigns, the impact on their beliefs can be profound and deep-seated: they rarely exit that rabbit hole.

But the authors also posit that misrepresentation using VR may be far more impactful than ‘traditional’ social media because, drawing on general research into VR behaviour change, “the medium provides an experience, one that skews closer to an actual one than a mediated one, perceptually and psychologically.” The authors give a striking (and potentially controversial) study of an AI simulation of a confrontation between Israeli soldiers and a Palestinian couple (the woman was visibly pregnant) at a military checkpoint in which the soldiers eventually point their rifles at the couple when the Palestinian man reaches into his pocket. The researchers, conscious of the need to capture the nuances of this intractable conflict on both sides, say they took great care to ensure that the scene “reflects the realities of the ongoing Israeli-Palestinian conflict and the friction and tensions that culminate in military checkpoints”.

In this experiment, Israeli soldiers who embodied the Palestinian avatar demonstrated consistent empathetic attitudinal changes compared to control conditions, by making fewer “shoot” decisions when confronted with ambiguous vignettes, and controlled to display more empathy to the Palestinians when retested 5 months afterwards.

So, in short, while VR misrepresentation may require more sophisticated, well resourced effort, it is also can be that much more effective.

What is VR misinformation more potent?

In order to understand how the features of VR can give rise to misperceptions, the authors drew on the concept of ‘affordances’, which means the potential actions which a technology feature allows an end user to do. For example, on social media, the ability ‘the affordance of anonymity’ in online discussion forums may encourage users to share more personal information than they would in face-to-face interactions and to be ruder.

The authors looked separately at the affordances from the immersive features of VR and from VR content.

Immersive features of VR

Some (but not all) of the immersive features of VR discussed by the authors:

• Field of view (how much a scene a person can see at once). Research shows that large screens (those big wall mounted TVs), compared to small screens, are “67% more likely to cause processing that is susceptible to heuristics, that is, persuasive strategies that avoid central processing of messages.” While VR puts you in the midst of a large screen view, the authors posit the impact could cut both ways. If the VR was about ballot rigging, a large horizon view might de-emphasise the ballot box and give a wider context of the number of surrounding people supervising against the possibility of ballot stuffing. But a ‘flat earther’ might be convinced that a wide horizon screen view with no obvious curvature of the earth proves their point.
• Spatial audio (use of multiple sound channels to replicate proximity to the listener): spatial audio may do little to reinforce a flat earther’s willingness to believe what the field of view ‘proves’ to them. But in the ballot box stuffing example, even with a wide field of view, someone whispering into a user’s ear from behind about how you should take some of the ballots out of the ballot box, or the low murmurs from the crowds of voters, can reinforce the immersive experience of the misrepresentation.
• Haptic feedback (simulation of a tactile experience by vibrations in a user’s hand controllers): research shows that when used to simulate touching a human, such as a handshake, this can influence persuasive outcomes, such as purchase decisions.
• Olfactory cues: olfactory replication devices are now light and portable enough to be attached to VR headsets. Research demonstrates that scent was roughly equal in impact to seeing images, and more compelling than sound. In the ballot stuffing example, ballot worker can be paired with a scent of body odour for realism, in that since the worker looks and sounds nervous while stuffing the ballot box, he should also smell nervous.
• Above all else, tracking: as you swivel your head, the ‘surrounding’ scene changes. Research shows that this is so compelling because it hooks into how our brains have evolved to perceive, predict and instruct body movement in the real world. The authors give a ‘chase and hunt’ example: “imagine being in VR and spending hours on a detailed scene hunt where one is looking for evidence of fraud, and then finally finding a stack of uncounted ballots hidden in the back room.”

VR content features

The authors identify the following ‘content affordances’ of VR:

• Self-embodiment: VR allows you to walk in another person’s shoes, and should provide one of the most effective ways to “transfer visceral emotions from the experience to the user [and which] can translate into changes in behavior, attitudes, and emotional responses during the VR experience, as well as after the user has left the virtual environment.” While that can be a force for good, the authors note that in the Israel-Palestinian VR study “one can easily imagine slight edits to the scene to facilitate more nefarious uses of such a powerful tool.”.
• Persistence and consistency: the ‘realness’ of the VR world is reinforced by the consistency of the shared experience –if you come back at different times of the day, the shadow and lighting will have moved. The authors note that some VR platforms allow a scene or place, such as a restaurant, to be either public or cloned to be limited to a certain group of users, raising the possibility of 3D ‘echo chambers’!
• Social actors: misinformation on social media operates within the fairly simple, confined dimensions of text or images. However, in VR, the avatar can engage a much wider range of forms of communication: avatars who maintained eye contact are more likely to elicit agreement to a proposed policy. Human users can have trouble distinguishing which avatars are computer-controlled and which are human-controlled. This creates the potential for a malicious actor to ‘populate’ a VR scene with computer-controlled avatar to create the appearance of consensus around a particular action or policy, which is a key lever for misinformation. Avatars appearances also can be altered to elicit stronger ‘bonding’ from a user: Research on faces has shown that politicians whose faces have been slightly altered to share cosmetic similarities to the targeted voter leads to influence.

What can we do to address VR misinformation?

While social media misinformation remains a vexed issue, research has found that the number of US voters exposed to misinformation in presidential campaigns dropped from nearly half in 2016 to a quarter in 2020, suggesting that the social media companies are getting some traction in their efforts to control misinformation.

The authors note that there is lot of variation between VR providers in how they manage VR misinformation. Meta’s policies for the Horizon Worlds platform is amongst the most robust. Meta has a code of conduct enforced by human monitors who don to headsets to maintain order and remove malicious users.

However, the authors raise concerns that misinformation approaches adapted from the simpler world of social media will be less effective in the VR world:

“Real-time misinformation spread in VR is enhanced by an arsenal consisting of nonverbal cues, spatial audio, haptic touch, social conformity, and other affordances that all enable a more persuasive user-to-user message, as opposed to simple text, video, or images alone. …the ephemeral (at the individual level) and multimodal nature of the medium presents identification and observation challenges that current automated trust and safety mitigation techniques may struggle with.”

On the other hand, the richer data environment of VR also might provide scope for automated tools detecting misinformation: “3D models are represented symbolically, the size and shape of objects, the distance among objects, and the orientation of objects (i.e., gaze direction of people) are all potentially available as high-level labels within the files themselves or able to be inferred from the environment.”

The VR companies and the industry also have a role to play in building trust and security, including through cross industry collaboration in developing standards. But the authors also acknowledge, at the end of the day, managing VR worlds comes down to individual users – which given the enhanced capabilities of VR to create ‘make-believable’ misinformation is scary prospect