"Into the Twilight Zones": An ArtsWest symposium on virtual and augmented reality

Above, a video installation of Jennifer Steinkamp's "Still-Life 1" (2016) outside of the auditorium at the Cantor Center for Visual Arts. When viewing the actual piece, it is obvious that the fruit is anything but still - it floats on the black screen in constant motion, like much of the computer-generated work by the media and installation artist. The  exhibition will be on display at Cantor until July 2020, made possible with support from Mr. Jeffrey N. Dauber and Mr. Marc A. Levin. Photo by Andrew Brodhead.

On Thursday, February 27, 2020 at the Cantor Center for Visual Arts, the Bill Lane Center for the American West hosted its most recent ArtsWest symposium, “Into the Twilight Zones: Art and Virtual/Augmented Reality in the West.” The event, produced by Bill Lane Center Affiliated Scholar Marc Levin, brought together artists, experts and storytellers working in the fields of augmented reality (AR) and virtual reality (VR). In individual presentations and a panel discussion, the speakers explored the potential of these new media to enhance how artists and audiences both create and experience art.

Professor Jeremy Bailenson delivers the ArtsWest
keynote presentation, "Virtual Becomes Reality."
Photo by Andrew Brodhead.

To launch the event, Professor Jeremy Bailenson, founding director of Stanford’s Virtual Human Interaction Lab, introduced some powerful use cases for AR/VR, which included deploying the technology to increase empathy for both people and planet. Bailenson started with an explanation of a central AR/VR, principle, that of “body transfer”: “If you move your body and see an avatar moving with you at the same time, after about four minutes, the brain will treat that avatar body as if it were your own body. This is called ‘body transfer.’”

It turns out body transfer can increase empathy, according to research performed in Bailenson’s lab. The communications professor, who has a PhD in cognitive psychology, created an immersive virtual reality project to simulate homelessness. After using the technology to spend days (all virtually) in the life of a homeless person, subjects were more likely to support affordable housing policies. Similarly, Bailenson’s lab conducted research on how virtual reality impacts conservation behavior. During the experiment, wielding a virtual chainsaw to topple a giant redwood gave subjects an embodied experience; it allowed them to feel the vibration and noise involved in cutting down an actual tree, and then watch that tree fall to the ground. Perhaps owing to the personal responsibility participants then felt for the destruction, the study’s subjects later changed their behavior and began recycling more.

Bailenson shared an additional conservation experiment that demonstrated the power of AR/VR to change the minds of legislators in positions to make a difference for the planet. When Palau's reefs were decimated by climate change, Bailenson visited the island and created a virtual reality experience of the oceans. The powerful footage of coral reefs swayed Palau's lawmakers, who subsequently began supporting measures to conserve the fragile ecosystem.

VR Artist Jennifer Steinkamp demonstrates software she created that captures video of her talking in real-time, and then
transfers her silly or exaggerated facial expressions onto a large avatar face floating on the screen before the ArtsWest audience.
Photo by Andrew Brodhead.

The next presenter, Jennifer Steinkamp, a Los Angeles based media and installation artist, discussed her own process creating computer-generated AR/VR pieces that often begin with images from the natural world (trees, flowers), and manipulate those images using projection and animation. Steinkamp’s slides revealed the artist’s inventive, subversive, and playful style, which results in work that takes viewers to the edge of reality (as AR/VR technologies are particularly well-suited to do).  The world as rendered by Steinkamp is neither completely familiar nor completely unfamiliar. Her large-scale projections of swaying birch trees or floating pieces of fruit are a part of the audience’s lived experience while also inhabiting a space that is very much computer-generated, otherworldly, and slightly disorienting. Steinkamp closed out her talk by demonstrating software she’d created that captures video of her face in real-time, and then transfers her silly or exaggerated facial expressions onto a large avatar face floating on the screen before the audience. Again, this blend of the real and unreal epitomized how AR/VR can be harnessed to give viewers a more transcendent experience of their everyday lives.

Following Steinkamp was interactive storytelling expert Yelena Rachitsky, executive producer of experiences at Oculus/Facebook. Her talk, “Designing for Emotion,” focused on integrating AR/VR into storytelling practices to give viewers greater agency over and emotional connection to the work they are perceiving. In her presentation, Rachitsky proposed that in a VR project that is done right, audiences feel like the experience is actually happening to them, not happening to somebody else. After all, one’s proximity to the story is different with immersive technologies – viewers feel closer and more intimately connected to characters, and as a result, empathy increases. 

Particularly poignant was Rachitsky’s retelling of her own experience of “Carne y Arena,” a conceptual installation by Alejandro G. Iñárritu that explores the journeys of immigrants and refugees. The project takes audiences into a virtual space where they attempt to cross the U.S. border from Mexico with a caravan of asylum seekers; but the group gets caught. And while traditional films place audiences before a flat, rectilinear screen, distancing them with a proscenium and story that is meant to be processed in the mind, the immersive technologies used in “Carne y Arena” give viewers an embodied experience of being physically detained. Emotionally, this induces a state of fear, dehumanization, and trauma. “When we bring our body into account and an experience has sense memory, it’s more powerful,” Rachitsky shared.

Yelena Rachitsky, executive producer of experiences
at Oculus/Facebook, discusses the emotional
impact immersive storytelling can have on audiences.
Photo by Andrew Brodhead.

Another emotionally resonant and empathy-building project Rachitsky worked on, “Traveling While Black,” is a VR documentary exploring race relations in America. Again, this film employs VR technology to increase the proximity between the story’s subjects and its audience. The film centers on “the Green Book,” a Jim Crow-era, crowd-sourced guide that alerted blacks where they could safely stay, eat, and travel in a segregated country. Most of the documentary takes place inside Ben’s Chili Bowl, a black-owned restaurant in Washington, D.C., where African-American patrons share stories of their restricted movement, and the fear and humiliation that comes with “traveling while black.” According to Rachitsky, the latest VR camera used in the documentary allows the audience to feel physically very close to the subjects – as if they are in the restaurant themselves – listening to the experiences as they are recounted. As a result, viewers transform from passive observers to active participants, brought into intimate contact with the emotional accounts of racial segregation and discrimination. 

After presentations from a number of VR artists and practitioners, the symposium turned to the museum side. How are museums harnessing AR/VR technologies? Elizabeth Merritt, founding director of the Center for the Future of Museums, discussed museum applications for immersive technology, highlighting both the limitations and opportunities of AR/VR. On the one hand, the new technology helps museums boost imagination; extend their reach beyond the museum’s four walls; create impossible experiences; facilitate interaction with untouchable objects; and potentially provide an alternative to the brick-and-mortar museum. On the other hand, Merritt pointed out, AR/VR can be costly, high-maintenance, and could potentially erect economic barriers, by assuming, for example, that all museum visitors might have a smart phone or tablet with which they can access an installation.

A slide from Elizabeth Merritt's presentation shows an iPhone application layering dinosaur skin, muscle and facial features over its bones. Museums have been using this kind of immersive technology to "fill in the gaps" in audience's imaginations. 

As an example of the “imagination boost” application of immersive technologies, Merritt shared an anecdote about one of her visits to a dinosaur gallery that made the ArtsWest audience chuckle. She was looking at a large skeleton of a Tyrannosaurus rex when she overheard a little boy ask his father, “But Dad, when he ate, did the food just fall out?” As imaginative as kids are, sometimes they need a little bit more information to complete a picture that is already obvious to experts, Merritt explained. How did the dinosaur skeleton look when it was a real creature? Many museums have used augmented reality to fill in some of the blanks for visitors, as shown above in Merritt’s slide of an iPhone application layering dinosaur skin, muscle and facial features over its bones. Suddenly, the child was granted entry into a world he previously couldn’t fully perceive or understand.

To close the symposium, Stanford’s Camille Utterback, assistant professor of art and art history, moderated a discussion with all of the speakers. Since the evening focused on augmented and virtual reality in the American West, and was hosted by the Bill Lane Center, Professor Utterback asked the panel how the story we tell about AR/VR is part of the mythology of the American West. In other words, when people describe AR/VR as being a limitless new frontier, as having futuristic qualities that allow for body transfer, the hack of human perception, etc. – how does this narrative play into the larger mythology about the West as wide-open territory to expand into and inhabit? And how is this similar to the way people describe Silicon Valley as a place of constant progress and endless possibility? As perhaps expected, AR/VR evangelist Yelena Rachitsky commented on how, at least personally, immersive technologies have given her an opportunity as an artist and storyteller to create almost anything. The field does feel full of endless possibility to Rachitsky. And in spite of their acknowledgement of AR and VR’s limitations, the other panelists’ enthusiasm for the technology – its transcendent power to take humans so fully into other lives and worlds – suggests the “new frontier” mythology of virtual and augmented reality holds powerful sway over their imaginations too.