Technology Tools

VR and AR: Transforming Learning and Scholarship in the Humanities and Social Sciences

What if a text or work of art is no longer read, but instead, experienced? What new questions are raised when it’s possible to visit an author’s home or stroll through the streets of an ancient city? How will our interpretations of literature, art, history and archaeology change when we are no longer passive recipients but co-constructors and actors in immersive experiences? How will this challenge us to think outside our current learning paradigms? These and other questions arise when we examine the impact of immersive technologies on the humanities and the social sciences.

As part of a class at Boston College, students developed a virtual reality game based on James Joyce’s “Ulysses,” titled Joycestick. The game, which recreates elements of Joyce’s complex world, was developed in the digital humanities course taught by Joseph Nugent, associate professor of the practice of English. For Nugent, Joycestick represents new ground for the teaching of literature.

“As much as I love literature, my traditional training hadn’t offered me the kind of fulfillment that comes from the kind of cutting-edge research technology has made possible.”

The collaborative effort brought together 22 students in literature, history and computer science, not only from Boston College, but nearby Berklee College of Music and Northeastern University as well. Students storyboarded and coded the book into a VR experience, giving users the ability to interact with objects in the scene. As one of the students, Ryan Reede, a computer science major said:

“Instead of using virtual reality as just an extension of video games, we’re trying to use it as an entirely new medium of storytelling. . . There’s interactivity, there’s audio, there’s text, there’s music, there’s all these different elements to Joycetick, but at the end of the day, it’s more of a new form of storytelling.”

At Duke University, the Dig@Lab in the Department of Art, Art History & Visual Studies focuses on simulating the past instead of reconstructing it. Playing off its name in “digging for information”, the Lab collaborates with departments of Classical Studies, Computer Science and the Institute for Brain Science. In a Virtual Museums course, students created their own virtual museums, visualizing what has been lost due to conflicts in the Middle East.

In collaboration with Duke, the Università Iuav di Venezia and the Università degli Studi di Padova have recreated the church of the Eremitani in Padua. Nearly destroyed during World War II, they digitally scanned the reconstructed building and virtually restored the architectural detail. Through augmented reality using mobile devices, visitors can experience the space and the original works of art which up to now have only been available through two dimensional photographs.

This summer, Yale University’s Digital Humanties lab released John Ashbery’s Nest—a collaboration between Karin Roffman, senior lecturer in humanities, English and American studies, and a team of designers, photographers, and coders from the Yale University Digital Humanities Lab and Yale ITS. Through a selection of 360° images and interactive layers, the project takes you inside the poet’s former home in Hudson, New York. A little known collector of American antiques, Asbury filled his house with objects that often served as inspiration for his poems. Walking through the space, you encounter audio clips, photographs and archival resources that enriches the setting and context of his work.

Virtual reality is opening new ways for people to experience works of art and understand the minds of artists. In a special exhibit, Disney & Dalí: Architects of the Imagination, the Dalí Museum created a Dreams of Dalí experience using Oculus Rift headsets. No longer just a viewer of Dalí’s famous painting, Archaeological Reminiscence of Millet’s “Angelus”, you can now step inside the artist’s work. Moving among the towering figures in a surreal landscape, you experience the figures from different perspectives and discover elements of other paintings by Dalí. Instead of recreating a single work by the artist in 3D, the VR experience is a doorway into Dalí’s imagination.

This dynamic of immersion and participation can lead to new ways of experiencing archaeological projects. The Kivik Grave, Virtual Bodies in Ritual Processionis a collaborative project from the Department of Design Sciences, Lund University Humanities Lab, the Institute of Archaeology and Ancient History and the Museum of Österlen, Sweden. Drawing from Sweden’s most well-known Bronze Age grave images, visitors are challenged to understand an enigmatic ritual from the past. The interactive art work serves as a “Virtual Reality play”, turning the spectator into both an actor and interpreter of a ritual experience.

At the Newseum in Washington, D.C., a VR exhibit takes visitors to Berlin during the height of the Cold War. The project was developed through a collaboration between museum staff, HTC and the University of Maryland’s Institute for Advanced Computer Studies. In one of the scenes, you are standing in a guard tower and using a searchlight to catch wall jumpers the way East German soldiers did. In the final segment, you can pick up a virtual sledgehammer and help destroy the wall. As one 13-year-old student has said,

"I felt like I had so much power in that moment . . . This is a really important part of history that affected so many people's lives, and the entire experience felt like you were actually there helping to make a difference."

Instead a reading about an historical event, museum-goers become actors in the experience, feeling as if they were actually there.

Bringing the Humanities and Social Sciences into the Public Sphere

Drawing upon the sense of unmediated presence of VR, these projects bring people into the experience, turning them from passive readers into active participants, inspiring new thoughts and meaning. They allow multiple entry points into remote and challenging topics and offer new modes of interpretation for art, literature, history, and archaeology. Students and faculty are not just recreating moments and objects from the past, they are creating immersive environments, which support emotional engagement, empathy and situated cognition.

Unlike a book or a painting on the wall, these projects become immersive platforms for human experiences by creating new spaces for multisensory human perspectives. In remarkable fashion, they are engaging the broader public and extending the work of the academic community farther into the public sphere. VR and AR will mark a new era in the humanities and social sciences, opening new ways to understand our heritage and the rich complexity of human experience.