In this guest-produced episode, Kopal Srivastava and Morgan Hogenmiller dive deep into virtual reality applications. They discuss the social and cultural applications for VR with Tom Corbett, a CMU faculty member at IDeATe Lab, before experiencing the possibilities of VR art and concerts firsthand. After trying out the Soundscape VR and Tilt Brush applications, Kopal and Morgan discuss the ways VR technology might be used in the future and what changes need to be made in order for immersive experiences in the metaverse to be more comfortable and rewarding.
Victoria Sprowls 0:02
Welcome to the Tech in the Arts podcast for the Arts Management and Technology Lab. Today we have a guest podcast edited and produced by two AMT Lab contributors, Kopal Srivastava and Morgan Hogenmiller, who take a deep dive into the subject of Virtual Reality. They interview Tom Corbett, a faculty member at Carnegie Mellon’s IDeATe Lab - which specializes in the Intersection of Design, Art, and Technology - about the social and cultural applications for VR. Then, they experience the possibilities of VR Art and VR concerts first hand, discussing the opportunities and potential pitfalls of this popular technology. Enjoy!
Kopal Srivastava 0:39
Hey everyone, I'm Kopal.
Morgan Hogenmiller 0:40
And I'm Morgan. Welcome to our conversation about VR in art therapy and entertainment. Today, we will be talking about virtual reality from our perspective as master's students with interests in the technology’s diverse applications in public health and entertainment. We will talk through VR and its place in these industries and review our experiences, testing two relevant VR applications. We'll start by briefly discussing what is VR. VR's development history began as early as the 19th century with the discovery of stereopsis, or the brain's ability to perceive things as three-dimensional if they're presented at a certain angle. Fast forward over a century to the 1960s, and we saw a lot of research into productive applications of 3D imaging as the military began building flight simulations with newly created VR technology. In the 1990s, the military also began running experiments on the psychological impacts of immersive VR and PTSD victims recovering from combat-related trauma. The use of VR and exposure therapy treatment has expanded since the late 20th century to heal many other psychological disorders, and into the art therapy space as well as we'll touch on later. When it comes to widespread public VR adoption, the gaming industry has most successfully capitalized on the intrigue of VR in artificial environments. According to Steam, VR gaming sales grew 32% in 2020, and Valve also reports seeing 1.7 million new VR users on Steam last year. Corporate giants are finding ways to apply VR technology to all facets of life as well, with Facebook at the helm. Facebook's recent name change to Meta is the biggest current example of how VR is top of mind in the technology space. The company envisions a future where VR technology is pervasive in our work and social lives to the point that we exist in two universes: the physical and the metaverse. We borrowed a VR headset from Tom Corbett, a faculty member at Carnegie Mellon University's IDeATe Lab to conduct research on the technology firsthand. And he talked with us about some interesting social and cultural applications of VR that he has worked on at CMU's Entertainment Technology Center.
Tom Corbett 2:55
We were looking at VR as a modification of your home, either it could be if you're in a busy home environment, can you create like a calm meditative place? Or other use cases where you know you're a student at University - you're an international student at a University. What if you could slip on the headset and be back in your bedroom or your family room at home? Just as a means of kind of returning to a place that's important to you? What if you could set up cultural centers where you know, your family, people from your country - you're scattered around the world - your city or whatever, could come together and meet in a virtual environment that is culturally, like appropriate and crafted towards you.
Kopal Srivastava 3:39
Now, if we talk about the barriers to entry for organizations adopting VR, what's the first word that comes to your mind? Cost? Hardware issues? Well, schools like CMU businesses, hospitals, and art and entertainment organizations are working hard to find the right fit for VR within their ecosystems. We'd be remiss if we didn't talk about some of the barriers to using VR that apply to any organization that wants to implement it. Like I said, one of the most obvious barriers is the cost not only to the businesses or organizations but also to the consumers. The next is that VR technology is still cumbersome, and sometimes glitchy. Despite many iterations of VR headsets and features, they can still be extremely uncomfortable. And VR graphics is sometimes lacking and even nauseating. Tom filled us in on some details of his experience with VR technological progression. And here's what he had to say about it.
Tom Corbett 4:43
This, this laptop was the best form factor we could get to run VR four years ago, right? Four or five years ago. This has a GTX 1080 and you needed that much graphic power just to do something like Tilt Brush. You are generating content, you have to generate about three to four times the number of frames that you have to do for a traditional video game, per eye, which means you're really getting six to eight times. So first of all, there's just a lot of processing that goes into it. Second of all the sensors have to be able to read. It used to be you had these - you had to rely on these things. This was for the five - these are sweeping lasers signal that would then the devices would be able to read that signal triangulate their position based upon it. And now, the cameras and the processors have gotten good enough that with with five onboard four or five onboard cameras, computer vision can read your room and pretty accurately determine your position within the room.
Morgan Hogenmiller 5:42
VR technologies progression has provided a new digital space for artists and their audiences to connect in more meaningful ways. I'm interested specifically in where there may be applications for VR to impact communities through art and it's health and therapeutic benefits. Art therapy is a field of therapy that began in the 1960s when the National Institute of Health funded research on the psychological impacts of art-making on the critically ill. They found that patients creative expressions helped doctors get a better view into the recovery experience, and families cope with their illnesses, and the field has been expanding ever since. Now, some art therapists are using VR applications to facilitate therapy sessions. I am interested in the implications of this merger between art therapy and technology for arts organizations and their work building healthy communities. My primary research question is where are there applications for arts institutions to use VR in their community programming to make an impact on their audiences?
Kopal Srivastava 6:40
Yeah, no, I think it's really interesting that both our research questions revolve around such niche applications of VR technology. Yours has to do with art therapy, mine has to do with live entertainment, particularly live music. When it comes to my research question, it mainly boils down to how one can maximize the experience by incorporating VR tech. Now consider a scenario where one of your favorite artists is in town and the show is sold out. Or you know what, even if you could get a ticket, you would be so far away from the stage that you might as well just stay at home. So how do you manage to not compromise that experience, while also taking into account the convenience and economic factors? Now the entertainment industry is constantly evolving. And this is where VR comes in. And this is where my research comes in. What I found out is that, though convenient live streaming of concerts take away the essence of space and presence organizes a nod to selling concert tickets, they're essentially selling the experience. And while virtual concerts may not be a standalone solution to make concert-goers happy, immersive technology can really serve as an addition to the original concert experience. That could be from a financial perspective, or even from a health point of view, given the current pandemic. So given our shared interest in VR, Morgan and I decided to come together and conduct an ethnography to really understand not only our own research but each other's research as well. Now, what is an ethnography? An ethnography involves hands-on, on the scene learning and it is relevant wherever people are relevant. ethnography is the primary method of social and cultural anthropology but it is integral to the social sciences and humanities generally. And it draws its methods from many quarters, including the natural sciences. For each of our research, we chose two applications. One was Tilt Brush for the research that had to do with art therapy, and the one which had to do with live entertainment and music, we chose Soundscape VR, and we both took turns testing each other’s as well as our own applications. So Soundscape seems to be like a musical metaverse. Could you maybe talk about how digital it was?
Morgan Hogenmiller 8:58
I felt like it was more of a video game than a concert experience for me. So it felt very digital. I think just because it didn't replicate real life. For example, I was kind of expecting to go to a concert experience and have it look like maybe an auditorium or an outside music venue. But I was immediately put into like this galactic universe in soundscape VR where I could shoot stars and comets while I was listening to the music and fly around. And I think that just made it seem completely outside of my schema for what a concert is to the point that it felt very digital. And I'm not sure if you can adjust that for the type of concert. I think it would have been nice to have, for example, if I'm going to Billie Eilish, an experience that looks like an actual Billie Eilish concert and not this intergalactic world because I think that just doesn't really match up with my idea of a concert and would probably affect my experience. So that just wasn't really what I was expecting and it did feel pretty digital.
Kopal Srivastava 10:10
So there was so many elements there. Like you said, there was shooting comets and whatnot. Did all of that make it like suffocating? Or did you feel dizzy at any point?
Morgan Hogenmiller 10:23
Yeah, um, I felt a little bit dizzy when I was doing the flying part. It's so funny to me that they allow you to do this flying component in this concert. Fun, but I did feel a little bit dizzy during that. And I think that's probably something that needs to be figured out in the future if this is going to be marketed as a replacement for concerts because I'm sure there are people that will actually get sick from using that. I typically can go on lots of different roller coasters and be totally fine. And I definitely felt the gravity aspect of when I was supposed to be coming down from flying, my stomach did a little flip. So I think that'll be something to watch out for in the future is if that gets better as iterations of VR headsets progress and as these applications get a little bit more sound with their graphics and movement components.
Kopal Srivastava 11:22
So wait, so you said that it wasn't like the concerts you've been to? And how you've experienced them? So would you prefer it if there were like people in that VR space? And adding on to the question, would you also prefer it if maybe people were there in the physical space with you experiencing the VR thing at the same time?
Morgan Hogenmiller 11:48
That's an interesting question. I did enjoy being able to converse with you and Tom when we were doing the experience because it made it feel more like a social experience for me. When I was in Soundscape VR, I was actually placed in this intergalactic world completely alone. And that was also a strange experience because when you're listening to music, you want to start dancing. And I actually kind of started moving and felt weird because I was alone, and it kind of killed - killed the vibe that was being created there. And so I think, typically, when you're going to concerts, you are in a social environment, and that was really lacking in the specific application. I think I would want to be able to talk to people in the environment. And also, if I could see them with me in the space, that would probably be the most realistic that we could get. And would probably make me feel more like this could be a replacement for a concert. Because right now, it seems like it really just maybe it gives me the music, in a live sense, if I'm able to be in it as the artist is actually playing, but it's still not quite going to measure up for me.
Kopal Srivastava 13:06
It was lonely.
Morgan Hogenmiller 13:06
Yeah, like it was it was lonely. You're right it was.
Kopal Srivastava 13:12
Yeah. No, that's, yeah, that's interesting. And when you talk about avatars being present there, would that be something that would work for you? Or would you actually like to see other people there?
Morgan Hogenmiller 13:28
You know, it's hard to really tell my, my gut kind of says that an avatar still doesn't fully help me replicate the experience of a concert. And maybe that's just because I really, really like music, right? And so going to concerts is such a special thing for me. And I think it would take me a while to get used to the fact that the digital concert's supposed to be kind of a replica for that. And the avatar, unless it looks - I don't know, I don't even know, if it looks really realistic, if that would do it, to be honest. I think there's just something about being in the actual physical space with an artist and with the crowd of fans that makes a concert what it is.
Kopal Srivastava 14:14
Do you have any comments on the graphics of the application?
Morgan Hogenmiller 14:20
Yeah, I think the environment that I was put into in Soundscape VR seemed a little bit random. Like I didn't really feel like I was in a concert environment. I know I kind of mentioned it was this like, intergalactic space, but there were just floating objects that I didn't feel like really made me feel contained in any certain place. There was like a couch that would float by at one point and like a bunch of TVs and maybe like a waiter or something, and it just didn't feel cohesive enough for me to feel like I was really comfortable. I was kind of constantly trying to figure out like, where I should situate myself and really what I was supposed to do. Also being the only person in that environment was a little bit disorienting too. So I think that could have been better to create a more comfortable and just kind of easy-to-use environment I guess. The next application that we tested was Tilt Brush, which is one of the first painting VR applications built by Google in 2016. The application equips its user with a virtual paintbrush and immerses them in their art, providing minimal graphics for the user to build upon. We chose this particular application because it has been used in multiple clinical studies on virtual reality and art therapy and we wanted to step inside the artist's perspective to see the tools patients used in these experiments to communicate firsthand. Here's a snippet of Kopal's review on the Tilt Brush experience.
Kopal Srivastava 16:03
So for context, there were a lot of different environments that you could paint in, as well and I found myself exploring each and every environment and each and every brushstroke or things like that. And there were something that was like prompting me on the screen as well like in the VR environment as well. So it was not at all difficult to get a hang of that. And in that way, I think the interface was very user-friendly.
Morgan Hogenmiller 16:28
If you felt like it was comfortable to use like that, and you enjoy the different environments, when would you use this tool if you had it readily available to you?
Kopal Srivastava 16:38
Honestly, I think it was just really interesting and exciting being in that environment. I mean, I remember being like, "Oh, my God, this is so cool." So I think just, I see it as a form of escapism, I can see myself being immersed in it, and just, I don't know, letting stress out or maybe trying to express myself or brainstorming or just - okay, maybe I might be angry and just like going like, "Hey, I just want to paint some walls, right." So yeah, I think I can see myself doing that. But at the same time, maybe not for a long period of time, maybe like 10 minutes, I think that would be a good amount of time that I could set aside for something like this. After that, I would just feel tired really.
Morgan Hogenmiller 17:28
Yeah, that's interesting that you feel like there's a time limit almost to the tool, do you think it would be different if you could use the tool with other people?
Kopal Srivastava 17:37
I can definitely see a collaborative aspect involved with the brush. Like I mentioned before, there are like a lot of brainstorming things that could come out of it. And people can play games, if they're in the same room, stuff like that. But it can also be pretty chaotic if you're there with like a lot of people because you are still like figuring out things on your own. And at least personally, I felt like having the VR headset on. It can get a little, like nauseating. And I don't want to experience that with other people as well. Like, it would just be like, "Okay, where am I? Who's here what to do?" So I can see myself in that space with other people. But then I guess the time limit would be even less that way.
Morgan Hogenmiller 18:33
Interesting. How do you think making art in a physical form differs from doing it in VR, you talked about the nausea that you get from the headset, do you think you would have a preference for one or the other?
Kopal Srivastava 18:50
I think just the excitement of doing it differently, would be overpowering enough to want to actually be in that space to create something. And of course, when it comes to doing it in the most organic way, like taking a pen and paper and just sitting and doing it that's a different experience, I agree. But it's like a blank canvas that's in front of you when you're in that VR space. And I would definitely like doing that. And I don't have a preference per se. But, just because it can get a little tiring, having a longer art piece being formed in a VR space might not be the best idea. So before actually experiencing Soundscape VR, I kind of jotted down my thoughts about how the experience is going to be like and I had a feeling that a concert VR are experienced will go through that initial moment of, "Oh wow, this is really cool. And this is really new," before actually diving into what's happening in the VR sphere. However, after experiencing Soundscape VR, I think Morgan and I both share a similar type process when it came to the bias that was associated with the experience, in that it wasn't positive but was actually negative. We both had very clear descriptions of how we expect a concert experience to be like, which is why once we were in that musical metaverse, the digital aspect and the VR aspect of it actually took away from the experience. On the basis of what I've read on VR concerts and reviews, the thing that will stand out the most is the ability to be actually in front of the stage while hearing your favorite artists perform. However, the thing that here comes in is the depth perception which, actually, again, my takeaway from the experience. The fact that a live experience gets you to actually interact with people in the arena is something that cannot be replaced. Moreover, like we talked about the heavy equipment associated with VR tech, it especially plays out to be an obstacle in such experiences. Having a VR headset on for hours, especially during the length of a concert can be painful and exhausting. This can be resolved with advancements in technology that deal with just that. This is where the world of AR comes in. However, all that being said, there will definitely be a surge of growth in VR and AR given the current pandemic.
Morgan Hogenmiller 21:46
Before experiencing Tilt Brush, I was curious about its graphics and potential benefits outside of art therapy. I thought collaborating in the tool might be a great way for users to share their life experiences freely through pictures and increase empathy. After comparing my and Kopal's experiences with Tilt Brush, I feel differently about the application and its potential uses. Both of us felt that Tilt Brush was very appealing in its novelty, and we enjoyed the freedom that it gave us to express ourselves and take over an entire space with our art. However, we felt that this excitement only lasted for maybe 10 to 15 minutes and that collaborating in the tool would not have extended this. We ultimately felt that making art in the physical world would be more enjoyable. If art-making applications are used in community programming, they may need to be more structured to keep their audience's attention. Or there may just need to be a time limit on their use until the wearable technology is more comfortable. The average time gamers spend in VR is about 32 minutes, so this could act as a good benchmark. Our experience with painting in concert simulations in VR left us pondering the importance of making and experiencing art in the physical world. We took away from our ethnography that VR can not yet match the enjoyment and peace we get when we spend time engaging with the arts live and in person. However, art-making in concert VR applications can act as temporary replacements fun and exciting spectacles and useful distractions from the real world to stimulate creativity.
Kopal Srivastava 23:09
So that was it for today. Both of us had a great time having this conversation and we hope you did too. Thank you all for tuning in.
Victoria Sprowls 23:20
Thanks for listening to the AMT Lab podcast and don’t forget to subscribe, wherever podcasts are found. Let us know what you thought by visiting our website amt-lab.org. That’s A-M-T dash L-A-B .org. Or, you can email us at amtlabcmu@gmail.com. You can also follow us on Twitter @Tech in the Arts, or on Instagram, Facebook, and LinkedIn at Arts Management and Technology Lab. Thanks again for listening. If you found this episode informative, educational, or inspirational, then send this to another arts aficionado in your life.