In this episode of the Arts Management and Technology Podcast: Tech in the Arts, Luna Lu speak with Eric Williams about the evolution of storytelling across emerging media, including VR, AR, and immersive experiences. Drawing on his work in cinematic virtual reality and the Cine-VR Lab, Eric explores how creators must adapt their narratives to different mediums, how immersive storytelling reshapes audience engagement and emotional connection, and how tools like AI are being integrated into creative workflows. He emphasizes the importance of choosing the right medium for each story and reflects on the future of storytelling as both a technological and deeply human practice.
Show Notes
Virtual Reality Cinema (or cine-VR)
Transcript
Eric Williams
I think the most important thing to keep in mind as a storyteller is to keep two thoughts in your mind at the same time: what is my story and what is my media, right? And so just to take it to traditional media, if you and I were going to collaborate on creating a story, we would write the story differently if we knew it was gonna be for radio or for podcast versus if it was gonna be for television or film.
Luna Lu
Welcome to another episode of Tech in the Arts, the podcast series from the Arts Management and Technology Laboratory at Carnegie Mellon University. My name's Luna Lu, the technology manager and podcast engineer at AMT Lab. Today I'm excited to be joined by storyteller, writer, and immersive media creator Eric Williams as part of the SONA Immersive Storytelling Festival.
SONA is an annual festival at Carnegie Mellon that brings together filmmakers, artists, and researchers to explore the future of storytelling through immersive media like VR, AR, and interactive experiences. Before we begin, I like to share a little background for our listeners.
Eric is a director of the MFA in Communication Media Arts Program. A professor in the McClure School of Emergent Communication Technologies at Ohio University's College of Communication. He also serves as a cinematic VR lead at the Game Research immersive design lab. His work spans screenwriting, immersive storytelling and emerging media with a focus on how story structures involve alongside new technology, how creators can adapt their stories into new formats and audiences.
Thank you so much for being here today. I'm really looking forward to talking about your journey as a storyteller, how technology is changing the way we experience narrative, and what the future of immersive storytelling might look like. To start us off, what was the moment you realized you wanted to move beyond traditional film into more immersive storytelling like VR? How did that transition happen?
Eric Williams
I was really pushed out of the nest. I think I was teaching screenwriting and traditional filmmaking, and a colleague of mine, this is about eight or nine years ago, Ohio University was giving away four $1 million grants to faculty who wanted to bring new ideas to the university.
And my colleague John Bowditch, who was running the grid lab, the game research and immersive design lab, got a group of us together and said, let's, VR is gonna come back. So why don't we make a pitch to Ohio University that we will shepherd VR back? And so my first response was, I'm not a tech person, there's no way. And he said, no, there's this thing called 360 degree video or cinematic virtual reality. It's right in your wheelhouse and just help me write the proposal. Eight months later we got the grant. They gave us a million dollars and I was, I was stuck. So I had to give it a shot and I really started to fall in love with it as a new medium.
Luna Lu
Awesome. So how should future creators think differently when writing stories for immersive media versus traditional film? 'cause you toss screenwriting in traditional filmmaking, right? So how does that differ really in your opinion? Or if it doesn't, right?
Eric Williams
I think the most important thing to keep in mind as a storyteller is to keep two thoughts in your mind at the same time: what is my story and what is my medium, right? And so just to take it to traditional media, if you and I were going to collaborate on creating a story, we would write the story differently if we knew it was gonna be for radio or for podcast versus if it was gonna be for television or film. And so the more you work in a medium, the more you start to figure out, maybe this is the type of story that I can tell better in radio, or this'll work better on television, or, you know, I've got an idea for a video game, it won't, it wouldn't work as a television episode. It'll work great as an interactive game. So I think the more you work in a medium, if you start working in cinematic virtual reality, or if you start working in augmented reality, I think at first you just have to dive in and try to tell the same stories, this is different in augmented reality. Oh, that starts to affect the storytelling brain a little bit differently, and you start to figure out, oh, an immersive story might lend itself better to these emotions or these thoughts. So I think that's what a good storyteller does, is what are my tools and what can those tools help me say in the best way.
Luna Lu
Well, thank you. It's so interesting because technologies are emerging and there's so many new mediums coming out, so it'd be interesting to see how people fit those stories into the different mediums. Going with that, can you tell us a little bit more about what Cine-VR lab is and what problem you were trying to solve when you first created it?
Eric Williams
Sure. I don't know if we had a specific problem that we were trying to solve, but there's this technology, it shoots with a camera that records in every direction, and you can watch it on your computer, but I think the best way to watch a Cine, we call it Cine-VR for cinematic virtual reality. The best way to watch Cine-VR is in a headset.
And when you put the headset on, you can look in every direction as if you're in that location. And so it really puts your head where the camera was and so to kind of go back to my previous answer, our first year or two of telling these stories, we really tried to, we ended up trying to tell film ideas through this other medium, but then you start to realize no part of it. Is this an immersive quality? I can put you in the middle of a cocktail party, for instance, and depending on where I look, it will draw me into certain conversations or I'm going to put you at the scene of a crime. Allow you to inspect the room and you and I might put the headset on and what captivates your eyes and your mind might be very different from mine.
And so you could be completely invested in the northeast part of the room or the northeast part of the party, and I'm gonna look in the Southwest. And so we've been put in the same experience, but we've experienced it very differently. And all of a sudden that started, I mean, you were asking a question. It starts to raise a variety of different questions.
Luna Lu
Thank you. And given that it's such an immersive experience and the audience has more control over what they see in VR experience, right? So how do you guide attention without the traditional camera techniques?
Eric Williams
The way that you guide people's attention in Cine-VR is both similar and different from film. So in film, you're right, we're getting rid of the frames, but there are still frames. I mean, your audience can't see this, right? But you are sitting in front of a sound foam piece, right? But you're framed in a blue frame and so from my perspective, if I put the camera here, you are framed. It's not a traditional film frame.
So we start to find locations where we can block people's door frames. And what this does is to answer your question specifically. How do we guide people's attention? Well, people are drawn towards color, drawn towards things in frames that are framed nicely. And so if I can put you into a costume or an outfit that draws my attention to you because of color or contrast, and then I can put you in a frame, I can guarantee that I'm going to pay more attention to that character than the character who doesn't have as much contrast and isn't framed.
By blocking, I can bring you closer to the camera. I can take you further away. So that's a lot of the traditional filmmaking techniques that we still use on a different level than we have what's called audio swing. So for instance, we mic everybody. And so you could, let's say you're at a cocktail party and you have 15 people standing around talking.
Well, by lowering the conversation of 13, of those 15 people. If the two people at the north part of the room, all of a sudden their volume comes up, well, I'm gonna pay more attention to them. If you subtly brighten their color or increase their contrast and subtly so that the audience doesn't recognize it, all of a sudden we're kind of rewarding them.
See, they're easier to identify. They're more brightly lit. Their colors are popping a little bit more. I can hear them better. You give the audience the illusion that they're making that choice. Then when I want you to stop paying attention to them, I decrease their audio and maybe I bring somebody up on the other side.
They could still continue to listen to the people in the north part of the room, but it's a little bit easier to hear the couple over here. And so it tends to be what we call audio swing, it'll swing your attention to where we want you to look.
Luna Lu
Wow, there's so many little choices that can make a huge impact that the audience may not even be aware of, right? And thought was their own choices. And you mentioned in a previous talk that after VR experiences, people say, “remember when we were on the bridge” instead of, “remember that scene on the bridge? What does that shift from watching to experience mean for the creator's work?
Eric Williams
I don't know the answer to that. One of the things, you know, being at a university, people will always like, what sort of research are you doing? But I don't consider myself a researcher. I consider myself a storyteller. And so do you talk to, if you were interviewing Francis Ford Coppola, would you say, tell me the research you did about your directing of Godfather II?
It's like, no, you just directed this amazing movie. And he just knows that it works because people leave it and remember it and they like it. I think it's important to ask that question. I hope that people do try to study it, but I'll tell you the fact that people talk about the pieces that we make as if they were there. There's some sort of like real magic there. I mean, somebody will figure out why, but I'm not that person.
Luna Lu
Well, thank you for what you already contributed, right? Apart from that, how do you see immersive storytelling changing the industry in the next five to 10 years? And do you think the biggest barrier to VR, maybe technological or financial or just storytelling literacy.
Eric Williams
Okay, so let's unpack that. There's a couple things that I think are really important to distinguish. One is that not necessarily the technology that makes it immersive, and so I actually think, you know, there's a lot of cutting edge work being done in theater. It doesn't involve any technology, right? And so you, there's these immersive stories. You can go and participate and talk to the actors and they'll talk back as a character that is super immersive. And so I think we're approaching a tipping point where people might be getting either sick of technology or distrustful of technology. I just wanna really experience human entertainment or human pieces of art.
And so I think we might start seeing immersive storytelling through live actors. I think that's, I mean, that already exists, but I think that is going to grow. I think the second level of your question was how is the, how is immersive technology changing the stories that we're telling? I think that immersive stories hit us differently.
Here's one thing that I've been thinking about a lot about, and I wouldn't call it research 'cause it's more just me being artistic, right? But there's this concept in storytelling called primary and secondary emotions. So for the listening audience, the primary emotion is the emotion that the character is going through.
So we'll stick with co-pilot and we're watching The Godfather, Michael Corleone. We are watching him and he is angry. Now, the secondary emotion is how I feel about Michael being angry and he could feel angry and I could feel angry also. And so there, there's this simpatico relationship. But he could also feel angry and I could feel nervous, right?
Because I might have some sort of conscience that is bothered by Michael's actions and Michael Zang. And so that's part of the artists and the storytellers’ craft is figuring out when you want to merge distance first, primary and secondary emotions. I think in immersive storytelling that line becomes more gray right, because, oh, I'm trapped on the sinking boat with the character. Like I'm just as scared as she is. Like I, so there's this tug of war between where someone's brain is. I think a lot about is when do you have time to process it. In a film, you can be processing it while you're watching it, right? They do a slow dissolve, and now there's an establishing shot and you're thinking like, oh, I don't know if I would've killed half of the New York mobsters in one afternoon.
But in immersive stories, you're just in it. And so do you spend time processing it afterwards? And if so, is that story then continuing, right? I mean, if I'm still thinking about being on the sinking ship two days later, am I still in the story? Like I'm still, do I have storytelling PTSD, where I'm still emotionally connected to that emotion?
That's where I think there's a lot of room to explore. We have this thing called primo emo remo, pre-em, emotion, emotion and remembering the emotion. And you know, one of the things I really, I believe in is taking what's in an immersive story and then trying to bring it into the real world. If you're in a story, a fictional story, and you get some, let's just a token, right?
This coin we've made up as part of our story world. When you leave that story, if I told the story, well those thoughts and emotions are gonna stick with you. If I give you that token and you put it in your pocket, three days later, you reach into your pocket and there's that token. My hope is that it connects you back to that fictional story world in a real, in a very real way, and I think that's something that you just don't get. You could remember the Godfather, but you don't like reach into your pocket and pull out a cannoli. You're like, wait a minute, this reminds me of when I was in the backseat of this car. So I think there's something there that's really powerful for this as a new medium.
Luna Lu
The emotional attachment definitely feels different for VR movies versus traditional films. Right. And you kind of answered my question, but. Where are you using AI to fit in in your workflow? Is it a tool collaborator, or do you think it's a threat, or do you not use AI at all?
Eric Williams
So my thoughts on AI, it's such a deep pool of water to talk about. I think we're using the phrase AI generically. To age myself, right? So I'm in my late fifties. When I was in college, I was studying film and we cut film, right? And so there was this whole thing about. Digital technology, everything's gonna be digitized. So imagine if I asked you this question, what do you think of digital?
You think digital's gonna be the end of creativity? It's like, so AI, we use AI right now in a lot of our work, but we use AI in everything from getting rid of objects in the scene to, you know, we'll do quick ai, we need a, a radio announcer in the background. We're not gonna go hire someone, record it. I just need the radio announcer to tell me that it's three in the afternoon on a Saturday and like move on.
So as a tool, I think it's very easy for me to say, yes, we use AI, we embrace it. It's not a bad thing. If you start talking about AI as your actors or AI as your storytelling or your screenwriting partner, that's a different conversation. AI as the sole screenwriter, right? That's a different conversation and there's a whole, I'm reluctant to dive into that 'cause that conversation is very large and I go back and forth on it personally, do I?
Well, I do believe that, maybe not in my lifetime, but certainly in your lifetime, there will be systems that will allow you, I've probably two dozen screenplays full length and then probably a hundred short screenplays. Will I be able to take all of my work, put it into a system and have the Eric R. Williams screenwriting AI collaborator? I think that's gonna be highly likely, and I think that eventually it will do a good job. My students, I got into a big argument on Thursday and some of them were just adamantly against AI and they said it marginal, not marginalizes. It averages everything else.
Everything's just kind of middle of the road. And my response was, most of what you were saying I won’t necessarily disagree with. There will come a time where you will watch a movie that you like. What are you going to do when you find out that it was written by ai? You are going to see an actor that you are going to think is amazing.
What are you gonna do when you find out that he or she's AI there? There will be a tipping point in the next decade where amazing things are gonna be made and people are gonna be astonished by their AI. Now, I'm not saying that's good or bad, but I predict that is going to happen.
Luna Lu
Thank you for your thoughts. And that wraps up all my questions. Is there anything else you would like to share with our listeners?
Eric Williams
I think the most important part of storytelling isn't the technology and it isn't the medium. It's the story, and I think to go back to an earlier question, it's really important for us as storytellers to pick the right tool because stories are what make us human stories are what make us better. And so we need to figure out how to tell stories that will make the world a better place, and then figure out the best tools to do that.
Luna Lu
Well, thank you so much for joining us today, Eric. It's been a pleasure hearing about your journey as a storyteller and your insights on how immersive technology is reshaping the way we think about narrative and audience experience.
For our listeners, please check out the links in our show notes to learn more about Eric's work, Cine-VR Lab, and his projects exploring the future of storytelling.
Dr. Brett Crawford
Thank you for listening to this episode of Tech in the Arts. If you found this episode to be informative, educational, or inspirational, be sure to check out our other episodes and send this to another arts or technology aficionado in your life. If you want to know more about arts management and technology, check out our website at amt-lab.org, or you can email us at info@amt-lab.org. You can follow us on Instagram at techinthearts, or on Facebook and LinkedIn at our full name Arts Management and Technology Lab.
Our season runs from September through May, so feel free to follow us on your preferred podcast app, be it iTunes, Spotify, or YouTube.