Virtual reality and augmented reality technologies have been around for some time. There’s a substantial body of literature that’s worth reading to develop an understanding of how immersive media has evolved and the philosophical and media theoretical perspectives that preoccupied the early pioneers and continue to resonate today. Here you will find my annotated bibliography of relevant books, articles, and reports. Augmented Reality items are also included, however, the primary emphasis of this bibliography at this time is virtual reality. Some annotations are taken from paper abstracts or publisher book descriptions, most have been edited, however, claims of originality are neither expressed nor implied in this document (fair use is claimed, as this has been shared in an educational context). The goal is to eventually write a fully-original annotation for each item eventually. Your comments and suggestions for additional entries and corrections are welcome, you can get in touch with me using the contact form on this site. — David Tamés
Contents
Top five books
Hayles, N. Katherine. How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. University of Chicago Press, 1999, Amazon, WorldCat, Publisher
Hayles investigates embodiment in our contemporary techno-cultural landscape, weaving together three stories: how information became conceptualized as an entity separate from the material forms that carry it; the cultural and technological construction of the cyborg imagination; and the demise of the liberal humanist subject in cybernetic discourse. From these stories, Hayles asserts that we are now constituted in contemporary discourse as posthuman subjects. Much of popular culture addressing cybernetics (e.g. Blade Runner and I, Robot) and virtual reality (e.g. The Matrix and eXistenZ) reflect our anxieties of being constituted in a posthuman way, however, Hayles does not see posthuman as necessarily negative. If we conceive of human consciousness as embodied and part of a larger system, we may come to a more holistic view of nature and human experience. This could lead to a transformation of human destiny from an ethos of domination and control of nature to a more holistic understanding of the human experience and our interconnectedness with nature. Several critics have pointed out that Hayles does not provide a strong argument supporting her proposition of embodied cognition, there is quite a bit of neuroscience research that is consistent with her claims. That’s something that could easily be resolved in a second edition.
Heim, Michael. The Metaphysics of Virtual Reality. Oxford University Press, 1993, Amazon, WorldCat.
Just around the corner lurks full-blown virtual reality, in which we will be able to immerse ourselves in a computer simulation not only of the actual physical world but of any imagined world. As we begin to move in and out of a computer-generated world, Michael Heim asks, how will the way we perceive our world change? Heim considers this and other philosophical issues with an eye for the dark as well as the bright side of computer technology. As Heim writes, not only do we face a breakthrough in the technology of computer interface, but we face the challenge of knowing ourselves and determining how the technology should develop and ultimately affect the society in which it grows.
Laurel, Brenda. Computers as Theater, 2nd Edition, Addison-Wesley Professional, 2013, Amazon, WorldCat.
Henry Jenkins, the author of Spreadable Media, writes: “Brenda Laurel’s Computers as Theatre was one of the few truly transformative books to emerge in the heady, early days of the ‘digital revolution,’ demanding that we think of the computer as posing a series of creative problems that might best be addressed through the lens of the dramatic arts rather than purely technical problems that remain in the domain of the computer scientists. In this new edition, she revisits that classic text in light of her rich and diverse experiences as a designer, educator, and entrepreneur.“ Featuring a new chapter that takes the reader through virtual reality and beyond, this book presents a practical theory of human-computer activity and how the principles presented can help designers understand what people experience when interacting with computers. The book also describes how the participant’s enjoyment of a computer system should be the highest design priority..
Murray, Janet H. Hamlet on the Holodeck: The Future of Narrative in Cyberspace. Updated edition. The MIT Press, 2017, Amazon, WorldCat.
An updated edition of the classic book on digital storytelling, with a new introduction and expansive chapter commentaries. Murray writes, “I want to say to all the hacker-bards from every field — gamers, researchers, journalists, artists, programmers, scriptwriters, creators of authoring systems… please know that I wrote this book for you.” Hamlet on the Holodeck was instantly influential and controversial when it was first published in 1997. Ahead of its time, it accurately predicted the rise of new genres of storytelling from the convergence of traditional media forms and computing. Taking the long view of artistic innovation over decades and even centuries, it remains forward-looking in its description of the development of new artistic traditions of practice, the growth of participatory audiences, and the realization of still-emerging technologies as consumer products.
Ryan, Marie-Laure. Narrative as Virtual Reality 2: Revisiting Immersion and Interactivity in Literature and Electronic Media. 2nd Edition. Johns Hopkins University Press, 2015, Amazon, WorldCat.
Is there a significant difference between engagement with a game and engagement with a movie or novel? Can interactivity contribute to immersion, or is there a trade-off between the immersive “world” aspect of texts and their interactive “game” dimension? As Marie-Laure Ryan demonstrates in Narrative as Virtual Reality 2, the questions raised by the new interactive technologies have their precursors and echoes in pre-electronic literary and artistic traditions. Approaching the idea of virtual reality as a metaphor for total art, Ryan applies the concepts of immersion and interactivity to develop a phenomenology of narrative experience that encompasses reading, watching, and playing. In this Ryan reflects on the developments that have taken place over the past fifteen years in terms of both theory and practice and focus on the increase of narrativity in video games and its corresponding loss in experimental digital literature. Following the cognitive approaches that have rehabilitated immersion as the product of fundamental processes of world construction and mental stimulation, she details the many forms that interactivity has taken—or hopes to take—in digital texts, from determining the presentation of signs to affecting the level of story.
Articles for getting started
Laurel, Brenda. What Is Virtual Reality?,” Medium, June 15, 2016, www
Brenda Laurel, author of the landmark book Computers as Theatre, writes, “Virtual Reality is everywhere again, and that’s a problem. Almost immediately after the new trend began, people started shopping 360° immersive video as VR. It is not…” In this article, she clarifies what Virtual Reality is, and what is not. The taxonomy is important to understand. She argues that “When the term is appropriated, its meaning disintegrates. […] When we use the term just because it’s sexy, its meaning spreads like an oil slick over our media and dilutes it such a degree that we no longer know what it means?—?think ‘turbo’.”
Naimark, Michael. “VR Interactivity: Some Useful Distinctions,” Medium, October 17, 2016, www.
Naimark writes, “the word “interactivity” [is used] to mean several distinctively different things, each with different user experiences, different technologies, and different associated costs. If you understand these distinctions and you’re involved in VR, you will make better-informed decisions. This will be especially true understanding camera-based VR and its differences from model-based VR. […] The big goal here […] is to help the VR community develop a common language.” He also writes in this article, “Perhaps the biggest surprise in VR since Facebook acquired Oculus in early 2014 was how much cinematic, camera-based VR has stolen the show, at least in the eyes of the press and general public, even with its limited interactivity.”
McIntosh, Verity. “Virtually Useful: Step One – A brief history of VR,” Pervasive Media Studio, September 11, 2017, www.
For creative practitioners interested in making artistic content in VR, it can be confusing and difficult to know where to start. In this series of articles, Verity McIntosh shares what she and her colleagues have been learning about VR in terms of what it’s all about, the technology here now, and what we can expect around the corner.
McIntosh, Verity. “Virtually Useful: Step Two – What is VR/AR/MR/360?,” Pervasive Media Studio, September 19, 2017,www.
McIntosh, Verity. “Virtually Useful: Step Three – Design dimensions (what really works in VR),” Pervasive Media Studio, September 29, 2017, link.
Naimark, Michael. “VR/AR Fundamentals—Prologue: Science with Attitude,” Medium, January 26, 2018, www.
This is the first of a five-part series by Michael Naimark who has been involved in the field for a long time. He writes “The bottom line: VR hasn’t met its market expectations. “Maybe it took billions of wasted dollars and some deep breathing to reassess and recalibrate. Maybe AR will do better.” His technical report “Elements of Real Space Imaging” written while he was working at the Apple Multimedia Lab in San Francisco in 1991 provided a foundation for understanding the challenges of making images have a greater degree of realness or presence, and the importance of understanding the complex interaction between form and content. This series provides a significant update of this classic report and is recommended as fundamental reading for anyone working in AR and/or VR.
Naimark, Michael. “VR/AR Fundamentals—1) Audiovisual Resolution and Fidelity,” Medium, February 2, 2018, www.
Naimark, Michael. “VR/AR Fundamentals—2) Audiovisual Spatiality & Immersion,” Medium, February 9, 2018, www.
Naimark, Michael. “VR/AR Fundamentals—3) Other Senses (Haptic, Smell, Taste, Mind),” Medium, February 16, 2018, www. Naimark, Michael. “VR / AR Fundamentals—4) Input & Interactivity,” Medium, March 3, 2018, www.
Articles that shed light on the “VR as an empathy machine” controversy
Herrera, Fernanda, et al. “Building Long-Term Empathy: A Large-Scale Comparison of Traditional and Virtual Reality Perspective-Taking.” PLoS ONE 13, no. 10 (October 2018), https://doi.org/10.1371/journal.pone.0204494
Virtual Reality (VR) has been increasingly referred to as the ultimate empathy machine. however, empirical evidence supporting the claim that VR is a more effective method of eliciting empathy than traditional perspective-taking is limited. Two experiments were conducted in order to compare the short and long-term effects of a traditional perspective-taking task and a VR perspective-taking task (Study 1), and to explore the role of technological immersion when it comes to different types of mediated perspective-taking tasks (Study 2).
Kool, Hollis. “The Ethics of Immersive Journalism: A Rhetorical Analysis of News Storytelling with Virtual Reality Technology.” Intersect: The Stanford Journal of Science, Technology, and Society 9, no. 3 (2016)
This paper provides background on and raises ethical challenges in regard to virtual reality technology as a journalistic tool. The realistic and empathy-generating nature of 360-degree filmed news story experiences arguably changes the role and responsibilities of both the journalist and the viewer participating in the story. Using Clouds Over Sidra, an award-winning long-form VR documentary as the model for analysis, this paper examines how VR may change cultural communication. The influence and impact this new medium may have are important to examine for its impact on the way that subjects portray, journalists capture, and consumers learn about news.
McRoberts, Jamie. “Are We There yet? Media Content and Sense of Presence in Non-Fiction Virtual Reality.” Studies in Documentary Film 12, no. 2 (May 4, 2018): 101–18, https://doi.org/10.1080/17503280.2017.1344924
Sense of presence is a central but widely contested concept in virtual reality (VR) and has been the subject of significant debate, discussion, and research. Key factors considered to influence the sense of presence are media form, media content and user characteristics but as yet, relatively little consideration has been given to how these apply to the emerging field of non-fiction VR. Non-fiction VR can be distinguished from other forms of VR by trying to engage audiences with real-world stories, where a sense of presence is intended to offer audiences opportunities for empathic engagement and social transformation. This paper offers a framework to analyze how four media content dimensions (immersion, the positionality of the user, interactivity and narrative agency) influence sense of presence in non-fiction VR projects. With the intention of offering deeper insights on how a sense of presence relates to the purposes of nonfiction narrative, it unpacks each media content variable and illustrates how these can be applied critically in relation to two notable non-fiction VR works, Gone Gitmo [2007. Directed by Nonny de la Peña and Peggy Weil] and 6×9: A Virtual Experience of Solitary Confinement [2016. Directed by Francesca Panetta and Lindsay Poulton. The Guardian].
Moroz, Matthew, and Kat Krol. “VR and Empathy: The Bad, the Good, and the Paradoxical.” In 2018 IEEE Workshop on Augmented and Virtual Realities for Good, VAR4Good 2018, Institute of Electrical and Electronics Engineers, 2018, https://doi.org/10.1109/VAR4GOOD.2018.8576883
Virtual reality (VR) is cited as offering the ultimate empathy machine. This theory makes sense intuitively since VR enables a user to step into another’s shoes and experience the world as they do. We define this specific class of mental state as “emotional empathy”.The ability of VR to evoke emotional empathy is widely lauded as a good thing. In this paper, we invite labels such as “Luddites” and “technophobes” as we question the soundness of such claims. We instead offer warnings regarding employing VR is this manner and urge caution. Rather than dismiss the usefulness of VR in this realm we offer alternative implementation techniques in order to evoke more positive results in users.VR offers much utility for psychologists, psychiatrists, and neuroscientists due to the ability it affords to alter cognition. While promoting the medium in general, we offer warnings regarding potential short and long term neurological impacts. We encourage increased research focus on the underlying neural mechanisms that underpin VR’s successful multisensory hijack.
Moroz, Matthew, and Kat Krol. “VR and Empathy: The Bad, the Good, and the Paradoxical.” In 2018 IEEE Workshop on Augmented and Virtual Realities for Good, VAR4Good 2018, Institute of Electrical and Electronics Engineers, 2018, https://doi.org/10.1109/VAR4GOOD.2018.8576883
Virtual reality (VR) is cited as offering the ultimate empathy machine. This theory makes sense intuitively since VR enables a user to step into another’s shoes and experience the world as they do. We define this specific class of mental state as “emotional empathy”.The ability of VR to evoke emotional empathy is widely lauded as a good thing. In this paper, we invite labels such as “Luddites” and “technophobes” as we question the soundness of such claims. We instead offer warnings regarding employing VR is this manner and urge caution. Rather than dismiss the usefulness of VR in this realm we offer alternative implementation techniques in order to evoke more positive results in users.VR offers much utility for psychologists, psychiatrists, and neuroscientists due to the ability it affords to alter cognition. While promoting the medium in general, we offer warnings regarding potential short and long term neurological impacts. We encourage increased research focus on the underlying neural mechanisms that underpin VR’s successful multisensory hijack.
Sánchez Laws, Ana Luisa. “Can Immersive Journalism Enhance Empathy?” Digital Journalism, 2017, https://doi.org/10.1080/21670811.2017.1389286
Major news outlets such as the New York Times and the Guardian have recently launched ambitious immersive journalism projects. Adopting the technologies and rhetoric of immersive journalism first presented by Nonny de la Peña in 2010, these news outlets seek to use virtual reality and 360 videos to create deeper engagement and empathy with their audiences. Yet can immersive journalism enhance empathy? This question is unanswerable without a thorough discussion of the concept of empathy, a discussion that so far has been missing in the academic literature and popular commentary. This article addresses the gap by presenting current debates about the definition of empathy and using these debates to critically assess de la Peña’s immersive journalism projects ” IPSRESS ” and ” Hunger in Los Angeles, ” and the recent New York Times ” The Displaced ” and Guardian’s ” 6 x 9 ” immersive journalism projects. The conclusion is twofold: On the one hand, I will argue that some immersive journalism projects are approaching a format that may enhance empathy, and on the other hand, I will propose that the project of immersive journalism needs to go beyond this goal and into adopting a more forceful role in shaping the future of virtual reality.
Shin, Dong Hee. “The Role of Affordance in the Experience of Virtual Reality Learning: Technological and Affective Affordances in Virtual Reality.” Telematics and Informatics, 2017, https://doi.org/10.1016/j.tele.2017.05.013
As virtual reality becomes more and more mainstream, the role of affordances in virtual environments becomes an important question. The goal of this study is to explicate users’ motivational affordances and examine how they influence the acceptance of a virtual reality learning environment (VLE). It examines how motivational affordances in an educational virtual reality (VR) system affect user experience to track and achieve goals for users. A multi mixed approach was used by combining qualitative methods and a quantitative survey. First, a critical incident technique was used to explore a range of affordance factors related to VLE. Second, based on the affordance factors identified from the qualitative methods, a survey was conducted to examine the effects of an affordance on user cognitive processes and the influence of affordance on the learning process. The results of the user model confirmed the heuristic role of presence and immersion affordance regarding their underlying link to educational affordances, such as empathy and embodied cognition. The findings imply the embodied cognition process of VLE in which technological qualities are shaped by users’ perception and context. The results establish a foundation for VR technologies through a heuristic assessment tool from a user-embodied cognitive process. They confirm the validity and utility of applying affordances to the design of VR as a useful concept and prove that the optimum mix of affordances is crucial to the success of VR design.
Shin, Dong Hee. “Empathy and Embodied Experience in Virtual Environment: To What Extent Can Virtual Reality Stimulate Empathy and Embodied Experience?” Computers in Human Behavior 78 (2018): 64–73, https://doi.org/10.1016/j.chb.2017.09.012
This study investigates the user experience to clarify what it is like to experience stories in VR (virtual reality) and how immersion influences story experiences in immersive storytelling. This study explores the immersive storytelling context, developing and testing a VR experience model that integrates presence, flow, empathy, and embodiment. The results imply that users’ personal traits correlate immersion in VR: user experience in VR depend on individual traits, which in turn influence how strongly users immerse in VR. The way users view and accept VR stories derives from the way they envisage and intend to experience them. Rather than simply being influenced by technological features, users have intentional and purposeful control over VR stories. The findings of this study suggest that the cognitive processes by which users experience quality, presence, and flow determine how they will empathize with and embody VR stories. •The user experience of virtual reality storytelling.•How immersion influences story experiences in immersive storytelling.•Users’ personal traits correlate immersion in VR.•The way users view and accept VR stories derives from the way they envisage and intend to experience them.
Shin, Donghee, and Frank Biocca. “Exploring Immersive Experience in Journalism.” New Media & Society 20, no. 8 (2018): 2800–2823, https://doi.org/10.1177/1461444817733133
Although virtual reality (VR) has been widely used to deliver news stories in immersive journalism (IJ), it is not clear how people are actually experiencing these stories and their contexts. Focusing on the immersion feature of VR stories, this study explicates the user experience to determine what it is like to experience news stories in VR and how immersion improves viewing experiences in IJ. This study proposes a VR experience model in the IJ context that integrates cognitive, affective, and behavioral factors as the primary influencing determinants. The results indicate that the meaning of immersion strongly depends on the users’ traits and contexts and that the function of immersion is strongly determined by the users’ own cognition and intentions. VR stories are viewed and accepted based on the manner that users imagine and intend to experience them. The model demonstrates the users’ cognitive processes of experiencing quality, value, and satisfaction, which determine how people empathize with and embody VR stories. The results confirm the relationship between immersion and both empathy and embodiment, implying a new conceptualization of immersion in the IJ context.
Movies
Wachowski, Andy, and Larry Wachowski, Directors. The Matrix, Warner Brothers, 1999. http://whatisthematrix.warnerbros.com
Of all the movies that express our cultural anxiety about virtual reality, cyberspace, and posthumanism (e.g. Lawnmower Man, Brainstorm, Until the End of the World, Ready Player One, etc.) The Matrix stands out as one of the best along with Blade Runner. Computer programmer Thomas Anderson (Keanu Reeves) has lived a double-life as a hacker called “Neo” in the Net. Otherwise his life is relatively ordinary – in what he thinks is the year 1999 – until he is contacted by the enigmatic Morpheus (Laurence Fishburne) who leads him into the real world. In reality, it is 200 years later, and the world has been laid waste and taken over by advanced artificial intelligence machines. The computers have created a false version of 20th-century life – a computer simulation called the “Matrix” – to keep the human slaves satisfied, while the AI machines draw power from the humans. Morpheus has been searching his entire life for a “chosen one” to destroy the Matrix, and he believes Neo is it. Neo has his doubts, but through all his adventures with Morpheus and his crew, he starts to believe and joins Morpheus and Trinity (Carrie Ann Moss) in their struggle to overthrow the Matrix. Finally Neo, who is pursued constantly by “Agents” (computers who take on human form and infiltrate the Matrix), is hailed as “The One” who will lead the humans to overthrow the machines and reclaim the Earth. Source of this summary: The Cyberpunk Project (The Matrix)
Videos/Presentations
Murray, Janet H. “Thresholds of Reality: Creating Coherent Enchantment in AR,” AR in Action, 2017, https://youtu.be/PTHP6YqLcJE (video), https://inventingthemedium.files.wordpress.com/2017/06/ar-in-action-2017-murray.pdf (slides)
Janet Murray discusses the role of enchantment in the design of Augmented Reality experiences. enchantment is a sense of dwelling in an augmented space adjacent to the real world where contrary-to-reality things – things that we wish for or fear in real life – fill our senses, and we are given contrary-to-reality powers to create effortless transformations.
Additional books
Friedberg, Anne. The Virtual Window: From Alberti to Microsoft. The MIT Press, 2009.
As we spend more and more of our time staring at the screens of movies, televisions, computers, and handheld devices–“windows” full of moving images, texts, and icons–how the world is framed has become as important as what is in the frame. In The Virtual Window, Anne Friedberg examines the window as a metaphor, as an architectural component, and as an opening to the dematerialized reality, we see on the screen.
Youngblood, Gene. Expanded Cinema. E. P. Dutton, 1970, http://www.vasulka.org/Kitchen/PDF_ExpandedCinema/ExpandedCinema.html
A unique book that discusses both film and video art from the perspective of expanding technological change. It is perceptive considering its date of publication and covers early material well. It is still an important reference on pre-1970 computer-generated film and early American video art. A must-read for the serious student of video art. Youngblood shares with Russell Connor the credit of introducing the term “video art” into the critical discourse. Connor, who curated Vision and Television, a pioneer museum video art exhibition at the Rose Art Museum at Brandeis in January of 1970, used the term in his catalog. Expanded Cinema came out shortly after. Before that, not even the artists knew what to call it. (based on text from davidsonsfiles.org)
Grosswiler, Paul. Transforming McLuhan: Cultural, Critical, and Postmodern Perspectives. Peter Lang, 2010.
Transforming McLuhan explores the radical, humanist line of descent in interpreting Canadian media and culture theorist Marshall McLuhan’s work, rejecting the dominant view of McLuhan as a conservative, uncritical herald of technological determinism and capitalism. This McLuhan is the oppositional critic of modernity, resisting uncontrolled technological change, who seeks new media forms with a human face.
Lanier, Jaron. The Dawn of the New Everything. Henry Holt and Co, 2017
An inventive blend of autobiography, science writing, philosophy, and advice, this book tells the wild story of his personal and professional life as a scientist, from his childhood in the UFO territory of New Mexico to the loss of his mother, the founding of the first start-up, and finally becoming a world-renowned technological guru. Understanding virtual reality as being both a scientific and cultural adventure, Lanier demonstrates it to be a humanistic setting for technology. While his previous books offered a more critical view of social media and other manifestations of technology, in this book he argues that virtual reality can actually make our lives richer and fuller.
Levy, Pierre. Becoming Virtual, Reality in the Digital Age. Plenum Trade, 1998, https://www.researchgate.net/publication/30872223_Becoming_Virtual_Reality_in_the_Digital_Age
Pierre Levy takes a fresh look at the whole idea of what is virtual. He’s responding to the widespread belief, and sometimes even panic, that a digital society with emphasis on virtual interactions is necessarily depersonalizing. He takes particular exception to the notion that “virtual” and “real” are opposites. Instead, Levy argues that virtuality is one of four modes of existence, the rest of which he describes as reality, possibility, and actuality. Each is defined in terms of its relationship with its environment. In following Levy’s world view, you may find that he interprets some or all of those terms in ways you’re not used to, but the result is an interesting new approach to what it means to be part of an increasingly digital world. He examines the virtualization of several elements of our society: the corporal body, text, the economy, language, technology, contracts, intelligence, subjects, and objects. What he finds is not the destruction of the person so much as a transformation. Virtualization adds to but does not replace, the real, the possible, and the actual. By understanding what virtualization means and involves, Levy believes that society will gain a greater variety of options for interaction in all areas. Becoming Virtual is serious philosophical work, dense with ideas.
Mann, Steve, and Hal Niedzviecki. Cyborg: Digital Destiny and Human Possibility in the Age of the Wearable Computer. Doubleday Canada, 2001, http://wearcam.org/cyborg.htm
“Steve Mann is a cyborg. He sees the entire world, including himself, through a video lens–the WearComp system. He can control what he sees, liberating his imaginative space from the visual stimuli-billboards and flashing neon signs–that threaten to overwhelm us. While recognizing the danger that human beings could be controlled by technology and the corporations that produce it for profit, Mann is also fascinated by the vast possibilities presented by the wearable computer”–Back cover. Imaging the cyborg revolution — Owning the cyborg — Reinventing the cyborg — Shooting back: privacy in the cyborg age — The Mediated cyborg: toward community — The Right to think: imaging the cyborg community.
Rheingold, Howard. Virtual Reality. Summit Books, 1991
For many people, this journalistic account of Virtual Reality was their first introduction to this technology that from the very beginning the hype was decades away from possibility. Rheingold reported on research and futuristic vision applying virtual reality (VR) technology to create graphic images that simulate reality for the user.
Steinicke, Frank. Being Really Virtual: Immersive Natives and the Future of Virtual Reality. Springer, 2016.
This book focuses on the recent developments of virtual reality (VR) and immersive technologies, what effect they are having on our modern, digitized society and explores how current developments and advancements in this field are leading to a virtual revolution. Using Ivan Sutherland’s “The Ultimate Display” and Moore’s law as a springboard, the author discusses both popular scientific and technological accounts of the past, present and possible futures of VR, looking at current research trends, developments, challenges and ethical considerations to the coming age of differing realities. Being Really Virtual is for researchers, designers and developers of VR and immersive technologies and anyone with an interest in the exponential rise of such technologies and how they are changing the very way we perceive, interact and communicate within our digital society.
Turkle, Sherry, Editor. Simulation and Its Discontents. MIT Press, 2009
A collection of essays on the technologies of simulation and visualization and how they have changed our ways of looking at the world. Sherry Turkle examines the now dominant medium of our working lives and finds that simulation has become its own sensibility. We hear it in Turkle’s description of architecture students who no longer design with a pencil, of science and engineering students who admit that computer models seem more “real” than experiments in physical laboratories. Echoing architect Louis Kahn’s famous question, “What does a brick want?”, Turkle asks, “What does simulation want?” Simulations want, even demand, immersion, and the benefits are clear. Architects create buildings unimaginable before virtual design; scientists determine the structure of molecules by manipulating them in virtual space; physicians practice anatomy on digitized humans. But immersed in simulation, we are vulnerable. There are losses as well as gains. Older scientists describe a younger generation as “drunk with code.” Young scientists, engineers, and designers, full citizens of the virtual, scramble to capture their mentors’ tacit knowledge of buildings and bodies. From both sides of the generational divide, there is anxiety that in simulation, something important is slipping away. Turkle’s examination of simulation over the past twenty years is followed by four in-depth investigations of contemporary simulation culture: space exploration, oceanography, architecture, and biology.
Additional articles
Accenture. “Waking Up to a New Reality: Building a Responsible Future for Immersive Technologies,” White Paper, 2019, https://www.accenture.com/us-en/insights/technology/responsible-immersive-technologies
This report examines how extended reality (XR) — which includes virtual reality (VR), augmented reality (AR) and a growing range of other immersive tools — is already creating new value well beyond the worlds of gaming and entertainment. In addition to delivering enhanced customer experiences, immersive tools are being used to augment workforce productivity, provide training and deliver mental health treatments. According to this report, XR carries physical, mental and social risks requiring leaders to design, build, and deploy tools/business models responsibly.
Coyne, Richard. “Heidegger and Virtual Reality: The Implications of Heidegger’s Thinking for Computer Representations.” Leonardo 27, no. 1 (1994): 65–73.
This author addresses the assumptions that underlie most research into virtual reality (VR) and other interactive computer systems. These assumptions relate to tensions between views of perception as a matter of data input versus the notion of perception as mental construction. Similarly, there is a tension between the assumption that pictures are meant as representations of things and the opposing idea that pictures are meaningful as socially constructed human practices. Aspects of the philosophy of Martin Heidegger are invoked as a means of cutting through these dilemmas. This reading of Heidegger presents truthful representation as a matter of correspondence only when the truth is understood as a means of disclosing a world. The article concludes with practical suggestions for VR research and development appropriated from a Heideggerian perspective.
Davies, Char. “Changing Space: Virtual Reality as Arena of Embodied Being.” In The Virtual Dimension: Architecture, Representation, and Crash Culture, edited by John Beckman, 144–55. Princeton Architectural Press, 1998
The medium of “immersive virtual space” or virtual reality—as it is generally known—has intriguing potential as an arena for constructing metaphors about our existential being-in-the-world and for exploring consciousness as it is experienced subjectively, as it is felt. Such environments can provide a new kind of “place” through which our minds may float among three-dimensionally extended yet virtual forms in a paradoxical combination of the ephemerally immaterial with what is perceived and bodily felt to be real.
Davies, Char. “OSMOSE: Notes on Being in Immersive Virtual Space.” Digital Creativity 9, no. 2 (1998): 65–74, https://doi.org/10.1080/14626269808567111
This paper discusses the original artistic intentions behind the immersive virtual environment OSMOSE (1995). The strategies employed to manifest them include the use of an embodying user interface of breath and balance and a visual aesthetic based on transparency and spatial ambiguity. The paper examines the medium of immersive virtual space as a spatiotemporal arena in which mental constructs of the world can be given three-dimensional form and be kinaesthetically explored through full-body immersion and interaction. Throughout, comparisons are made between OSMOSE and conventional design approaches to virtual reality. The tendency of such approaches to reinforce the West’s historic devaluation of nature and the body is also discussed. It is suggested that this medium can potentially be used to counteract such tendencies. In the case of OSMOSE, an experiential context is constructed in which culturally learned perceptual/conceptual boundaries are osmotically dissolved, causing conventional assumptions about interior, exterior, mind, body, and nature to be questioned by the immersed participant.
Kane, Heidi S., Cade Mccall, Nancy L. Collins, and Jim Blascovich. “Mere Presence Is Not Enough: Responsive Support in a Virtual World.” Journal of Experimental Social Psychology 48, no. 1 (2011), https://doi.org/10.1016/j.jesp.2011.07.001
When individuals are faced with novel or threatening situations, the presence of a trusted companion should reduce anxiety and promote feelings of security. Attachment theory assumes, however, that mere presence is not sufficient for establishing security.
Kilteni, Konstantina, Raphaela Groten, and Mel Slater. “The Sense of Embodiment in Virtual Reality.” Presence Teleoperators & Virtual Environments 21, no. (4) October (2012), https://doi.org/10.1162/PRES_a_00124
What does it feel like to own, control, and be inside a body? The multidimensional nature of this experience together with the continuous presence of one’s biological body renders both theoretical and experimental approaches problematic. Nevertheless, the exploitation of immersive virtual reality has allowed a reframing of this question of whether it is possible to experience the same sensations towards a virtual body inside an immersive virtual environment as toward the biological body, and if so, to what extent. The current paper addresses these issues by referring to the Sense of Embodiment (SoE). Due to the conceptual confusion around this sense, we provide a working definition that states that SoE consists of three subcomponents: the sense of self-location, the sense of agency, and the sense of body ownership. Under this proposed structure, measures and experimental manipulations reported in the literature are reviewed and related challenges are outlined. Finally, future experimental studies are proposed to overcome those challenges, toward deepening the concept of SoE and enhancing it in virtual applications.
Lanier, Jaron. “Technology and Its Discontents.” Nature, September 18, 2014
Jaron Lanier surveys four studies probing the “vexed nexus of mind and digisphere”. Digital technology is remaking the cognitive environment in which human brains develop and function. This swift revolution is inevitably sparking much hard thinking. Books by neuroscientists Susan Greenfield and Daniel Levitin, and writers Nicholas Carr and Paul Roberts, propose either adaptation to the changes — self help strategies to compensate for emerging cognitive mis alignments — or critiques of the overall transformation.
McMahan, Alison. “Immersion, Engagement, and Presence.” In The Video Game Theory Reader, edited by Mark J.P. Wolf and Bernard Perron. Routledge, 2004
A recent shift in computer game design involves a move away from 2D level design in games like Prince of Persia (1992), or from isometric design in games like Warcraft, to 3D design and an frst-person point of view. This shift increases the sense of immersion by replicating the aesthetic approaches of frst-person shooter games in other types of games, such as adventure games, role-playing games, and even strategy games, which previously used 2D levels or isometric views. The shift in design is indicative of an overall trend to make desktop video games feel more like virtual reality. My approach here is to reexamine our concept of immersion in video games and suggest that immersion has become an excessively vague, all-inclusive concept. It is necessary to break down the concept of immersion into its more specific meanings and develop a more specific terminology.
Morie, Jacquelyn Ford. “Female Artists and the VR Crucible: Expanding the Aesthetic Vocabulary.” Proc. SPIE 8289, The Engineering Reality of Virtual Reality 2012, 828908 (8 February 2012), 2012, https://doi.org/10.1117/12.910974
A survey done in 2007 of VR Artworks (Immersive Virtual Environments or VEs) showed that women have created the majority of these immersive works. While this may seem counter to popular ideas that the field has been dominated by men, it seems rather, that the truly unique works appear to emerge from a feminine approach. Such an approach seems well suited to immersive environments as it incorporates aspects of inclusion, wholeness, and a blending of the body and the spirit. Female attention to holistic concerns fits the gestalt approach needed to create in a fully functional yet open-ended virtual world, which focuses not so much on producing a finished object (like a text or a sculpture) but rather on creating a possibility for becoming, like bringing a child into the world. Immersive VEs are not objective works of art to be hung on a wall and critiqued. They are vehicles for experience, vessels to live within for a piece of time. Virtual Reality was a technological wonder in its early days, and it was widely held to be a domain where men were the main practitioners. However, a survey done in 2007 of VR Artworks (Immersive Virtual Environments or VEs) showed that women have actually created the majority of artistic immersive works. This argues against the popular idea that the field has been totally dominated by men. While men have made great contributions in advancing the field, especially technologically, it appears most artistic works emerge from a decidedly feminine approach. Such an approach seems well suited to immersive environments as it incorporates aspects of inclusion, wholeness, and a blending of the body and the spirit. Female attention to holistic concerns fits the gestalt approach needed to create in a fully functional yet open-ended virtual world, which focuses not so much on producing a finished object (like a text or a sculpture) but rather on creating a possibility for becoming, like bringing a child into the world. Immersive VEs are not objective works of art to be hung on a wall and critiqued. They are vehicles for experience, vessels to live within for a piece of time.
Nash, Kate. “Virtual Reality Witness: Exploring the Ethics of Mediated Presence.” Studies in Documentary Film 12, no. 2 (2018).
The notion of immersive witness underpins much of the exploration of virtual reality (VR) by journalists and humanitarian organizations. Immersive witness links the experience of VR with a moral attitude of responsibility for distant others. In accounts of media witnesses, the ability of the media to sustain an experience of presence has played an important, albeit often implicit, role linking the spectator spatially and temporally to distant suffering. However, the concept of media witness has to date assumed that the media represent, that news stories and documentaries present to their audiences images and sounds that communicate something of an event. VR, in contrast, seeks to simulate, providing the audience with something of an experience that is linked in various ways to the experiences of others. It is this simulative function that is seen as fundamental to VR’s moral address.
Poster, Mark. “Postmodern Virtualities.” In Cyberspace/Cyberbodies/Cyberpunk: Cultures of Technological Embodiment, edited by Mike Featherstone and Roger Burrows. Theory, Culture & Society (Book 43). SAGE Publications Ltd, 1996
On the eve of the 21st century, there have been two innovative discussions about the general conditions of life: one concerning possible postmodern culture and even society, the other concerns broad, massive changes in communication systems. In this essay, Mark Poster contends that a critical understanding of the new communication systems requires evaluation of the type of subjects it encourages.
Sacau, Ana, et al. “Presence in Computer-Mediated Environments: A Short Review of the Main Concepts, Theories, and Trends Abstract.” In Proc. of the ADIS 2003, 2003
Computer-mediated environments present nowadays one of the key areas to the application and use of technology in work, leisure and education. It offers a number of opportunities to redesign the way we work, how we produce knowledge and collaborate with others. It also has some degree of influence in the way we manage information and perceive information generated by others. A number of e-applications that we must built need to be informed by a number of requirements that we can summarise as cognitive ones. Among these is the notion of presence. This paper presents an overview of the concept of Presence. This concept was proposed around 90s to describe “the sensation of being there” in computer-mediated environments. It offers one of the many opportunities to explain how engaged people are with the use of technology. The authors introduce the notion of presence in order to address their impact on applications and the way it can be found and be identified.
Shekhar, Shashi, Steven Feiner, and Walid Aref. “Spatial Computing.” Communications of the ACM 59, no. 1 (2015): 72–81, https://doi.org/10.1145/2756547
Knowing where you are in space and time promises a deeper understanding of neighbors, ecosystems, and the environment.
Slater, Mel. “Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments.” Philosophical Transactions of the Royal Society B 364 (2009): 3549–57, https://doi.org/10.1098/rstb.2009.0138
In this paper, I address the question as to why participants tend to respond realistically to situations and events portrayed within an immersive virtual reality system. The idea is put forward, based on the experience of a large number of experimental studies, that there are two orthogonal components that contribute to this realistic response. The first is ‘being there’, often called ‘presence’, the qualia of having a sensation of being in a real place. We call this place illusion (PI). Second, plausibility illusion (Psi) refers to the illusion that the scenario being depicted is actually occurring. In the case of both PI and Psi the participant knows for sure that they are not ‘there’ and that the events are not occurring. PI is constrained by the sensorimotor contingencies afforded by the virtual reality system. Psi is determined by the extent to which the system can produce events that directly relate to the participant, the overall credibility of the scenario being depicted in comparison with expectations. We argue that when both PI and Psi occur, participants will respond realistically to the virtual reality.
Slater, Mel, Bernhard Spanlang, and David Corominas. “Simulating Virtual Environments within Virtual Environments as the Basis for a Psychophysics of Presence.” ACM Transactions on Graphics (TOG) 29, no. 4 (2010): 1–9, https://doi.org/10.1145/1778765.1778829
A new definition of immersion with respect to virtual environment (VE) systems has been proposed in earlier work, based on the concept of simulation. One system (A) is said to be more immersive than another (B) if A can be used to simulate an application as if it were running on B. Here we show how this concept can be used as the basis for a psychophysics of presence in VEs, the sensation of being in the place depicted by the virtual environment displays (Place Illusion, PI), and also the illusion that events occurring in the virtual environment are real (Plausibility Illusion, Psi). The new methodology involves matching experiments akin to those in color science. Twenty participants first experienced PI or Psi in the initial highest level immersive system, and then in 5 different trials chose transitions from lower to higher-order systems and declared a match whenever they felt the same level of PI or Psi as they had in the initial system. In each transition, they could change the type of illumination model used, or the field-of-view, or the display type (power wall or HMD) or the extent of self-representation by an avatar. The results showed that the 10 participants instructed to choose transitions to attain a level of PI corresponding to that in the initial system tended to first choose a wide field-of-view and head-mounted display, and then ensure that they had a virtual body that moved as they did. The other 10 in the Psi group concentrated far more on achieving a higher level of illumination realism, although having a virtual body representation was important for both groups. This methodology is offered as a way forward in the evaluation of the responses of people to immersive virtual environments, a unified theory, and methodology for psychophysical measurement.
Slater, Mel, and Sylvia Wilbur. “A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments.” Presence: Teleoperators & Virtual Environments 6, no. 6 (1997): 603–616, https://doi.org/10.1162/pres.1997.6.6.603
This paper reviews the concepts of immersion and presence in virtual environments (VEs). We propose that the degree of immersion can be objectively assessed as the characteristics of a technology, and has dimensions such as the extent to which a display system can deliver an inclusive, extensive, surrounding, and vivid illusion of virtual environment to a participant. Other dimensions of immersion are concerned with the extent of body matching, and the extent to which there is a self-contained plot in which the participant can act and in which there is an autonomous response. Presence is a state of consciousness that may be concomitant with immersion, and is related to a sense of being in a place. Presence governs aspects of autonomic responses and higher-level behaviors of a participant in a VE. The paper considers single and multi-participant shared environments, and draws on the experience of ComputerSupported Cooperative Working (CSCW) research as a guide to understanding presence in shared environments. The paper finally outlines the aims of the FIVE Working Group, and the 1995 FIVE Conference in London, UK.
Stiegler, Christian. “The Politics of Immersive Storytelling.” International Journal of E-Politics 8, no. 3 (2017)
This article applies and extends the concept of social media logic to assess the politics of immersive storytelling on digital platforms. These politics are considered in the light of what has been identified as mass media logic, which argues that mass media in the 20th century gained power by developing a commanding discourse that guides the organization of the public sphere. The shift to social media logic in the 21st century, with its grounding principles of programmability, popularity, connectivity, and datafication, influenced a new discourse on the logic of digital ecosystems. Digital platforms such as Facebook are offering all-surrounding mediated environments to communicate in Virtual Reality (‘Facebook Spaces’) as well as immersive narratives such as Mr. Robot VR. This article provides an understanding of the politics of immersive storytelling and of its underlying principles of programmability, user experience, popularity, and platform sociality, which define immersive technologies in the 21st century.