By: Georgia Calawerts, Anjana Kankanala, Kody Proctor, Carolina Quintana, & Delaney Sears
INTRODUCTION
This project delves into the potential expansion in LED volumes, assessing both benefits and challenges. Our findings emphasize the impact of virtual production on labor markets, economic advantages of increased investment in LED screens, and the necessity of integrating virtual production across Sony's various entertainment divisions. Additionally, we highlight the significance of driving technological innovation to solidify Sony's leadership in this evolving landscape. The information revealed in this research, through a conducted survey, will further explore the current state of the virtual production industry and the creative evolution underlying its widespread adoption. For a deeper understanding of virtual production, please see Part I of this research.
Image: SONY Crystal LED Display System
Image Source: CineD
SURVEY
The survey focused on understanding the opinions and perspectives of artists and craftspeople working in the virtual production industry. The sample size for this survey was limited to industry and creative professionals with a solid working knowledge of the technology, as opposed to a larger consumer sample. The survey received 87 responses, with 47 completing every question. For consistency, data analysis was limited to these 47 Responses. Respondents included executives and creatives working in TV, motion pictures, video games, creative development, production, technical crew, and visual effects. Experience ranged from entry-level and mid-level positions to C-Suite executives. Survey responses were kept anonymous to maintain confidentiality surrounding potentially sensitive industry information.
RESPONSE ANALYSIS
Overall, respondents believe virtual production is a valuable tool for greater creative freedom and control over the final product. However, respondents also expressed concerns regarding the high costs associated with virtual production, the steep learning curve, the need for specialized technical knowledge, and gaps in technology. The survey results suggest that virtual production is seen as a promising development in the film and TV industry. Further research and refinement of the technology may be necessary to realize its potential more fully.
LED SCREENS AND ACTORS’ FOCUS
We asked respondents if they thought having a dynamic LED screen versus a green screen changed the impact on the actors’ focus during the performance, and the response was affirmative. The majority believed that LED screens increase actors’ focus (Figure 2). An example of this was in The Jungle Book (2016), when LED screens were used during filming to cast shadows on the child actor playing Mowgli to help him feel like he was not acting in a vacuum (Baker, 2016). Conversely, LED screens could also hinder actors’ performances, like in the case of Our Flag Means Death (2022), when using LED screens for a water scene in the background made some actors legitimately seasick (Schaefer, 2022).
Image: Figure 2: Respondents’ Opinion on the Impact of LED Screens on Actors’ Focus
Image Source: Created by research team
LED SCREENS’ IMPACT ON THE JOB MARKET
Though most respondents did not personally know anyone who had lost a job due to the emergence of virtual production, we asked how they assumed the emergence of virtual production would affect the labor force. Respondents ranked “below the line” as having the highest impact by the emergence of virtual production (pre-production, production, and post-production). There are also moderate-to-high concerns with the labor market adapting to new technologies. It has been pointed out in interviews and survey responses that the lack of knowledge about this technology can make it difficult for widespread adoption.
MOTIVATORS FOR LED SCREEN ADOPTION
Filmmakers must know the most important factors in deciding whether or not to use LED screens. When asked to rank what factors are most important, respondents gave the highest priority to cost-effectiveness, with secondary priority assigned to lower CO2 emissions.
Image: Figure 3: Respondents’ Top Factors for Choosing LED Screens
Image Source: Created by research team
THE CARBON FOOTPRINT
Lower CO2 emissions were one of the top motivators for adopting LED screens. When asked about how much respondents thought LED screens reduced the carbon footprint, most people correctly said 25%, but many respondents also said the number was outside the 5% - 30% range (Figure 4). From our client, SPE, it was stated that 25% was the closest approximation of CO2 reduction (A. Andrews, personal communication, January 11, 2023). According to the Producers Guild of America’s “Green Production Guide” report, the growing adoption of LED lighting for production has significantly reduced electrical demand on set while also revolutionizing the creative process (Sustainable Production Alliance, 2021).
Image: Figure 4: Respondents’ Views on LED Screens’ Impact on Carbon Footprint
Image Source: Created by research team
Survey responses showed a general lack of knowledge surrounding the environmental benefits of virtual production growth, but comparing virtual sets to on-location shoots in terms of CO2 footprint provides a realistic look at the ultimate environmental impact (Figure 5).
Image: Figure 5: Carbon Footprint Analysis, Virtual Production vs. Location
Image Source: Created by research team
Below is a figure showing the steep differences in the CO2 emissions and the offset costs of virtual versus location shoots over time. (Figure 6)
Image: Figure 6: CO2 Emissions & Offset Costs, Virtual Production vs. Location
Image Source: Created by research team
INTERVIEWS
For young filmmakers who prioritize environmental impact, these savings, both carbon and cost, could be a game changer. Expanding this understanding will become a pivotal point in later recommendations for studios seeking to convince more creators to incorporate LED screens into their projects.
CHALLENGES OF LED SCREEN ADOPTION
The following non-ranked responses were given in an open-ended question asking what limitations or technical issues could hinder the decision to use LED screens.
Insufficient budget/resources. (For example, accessibility to pre-visualization assets or high-definition models/assets, high Heating, Ventilation, and Air Conditioning (HVAC) costs)
Lack of technical crew that knows how to operate LED processors or properly color calibrate their LED surfaces, the software, or the hardware
Technical issues such as visual glitches, response lag time, and screens overheating
Size of physical objects required in the scene and number of performers
Limitations of wide shots to the stage size
Dynamic screens triggering epilepsy conditions in actors (the flashing lights could cause issues) or motion sickness
Candid responses like these provided insights into why some creatives might not want to use LED screen technology. One respondent elaborated on their understanding of LED screen limitations with the following:
“Scale. As of now [sic] the small enclosed space of something like the Volume means that you have very little space that's actually tangible. When used selectively, like in The Batman, this doesn't matter much, but when you're restricting too much of your world to this program you end up with a world that feels flat and hollow. Compare some [of] (The) Mandalorian scenes to how lived in and real Andor feels.”
We also asked if additional features or capabilities would make LED screens even more attractive to creatives. Responses included adaptation for larger-scale filming, facilities for lower-budget and student-run films, portability, and the ability for virtual environments to respond to and interact with physical props.
GREEN SCREENS: TO STAY OR NOT TO STAY
The most interesting insight from the survey is about the fate of green screens. In our talk with Eric Liebmann, he pointed out that many people in the industry believe that green screens will be a thing of the past in the next 10-20 years (personal communication, December 1, 2022). When asked if green screen technology will become obsolete in the next 10-20 years, 51% of our respondents said it would not, while the remaining balance of responses was split between ‘yes’ and ‘maybe.’ Those who maintained that green screens would not become obsolete cited the cost-prohibitive nature and lack of portability of LED screens as primary reasons.
EXECUTIVES AND DP’S AND EDUCATORS, OH MY!
The growth of virtual production involves multiple departments, so we sought different representatives of each industry sector to gauge their particular experience, including representatives from virtual production companies, video game companies, such as Oddworld; large studios, such as Paramount and Amazon; and more. All quotes were anonymized below. We launched five control questions for all interview subjects while adding custom questions that matched the interviewee’s position and experience:
Do you believe green screens will become obsolete and replaced by LED technology?
Who are the leading studios and players in the LED/virtual production space, and who are the emerging leaders?
How will the emergence of this new technology affect the labor force (job retention, job growth, training concerns, etc.)?
How is this new technology affecting story development, project green lights, and artistic considerations?
Who are the main competitors when it comes to virtual production training? Are there companies that have spearheaded efforts, or have universities led the charge?
The first company we interviewed was Sony. We interviewed the Executive Vice President and Chief Technology Officer of SPE, and Executive Vice President of Production at Sony Pictures TV. Their insights focused on Sony’s understanding of the technology’s current limitations. Both mentioned the immobility of Sony’s current screens, but both cited Sony’s recent acquisition of Pixomondo as a starting point for exploring more adaptable screens (EVP personal communication, January 26, 2023; EVP, personal communication, March 1, 2023). Their understanding of Sony’s current virtual production priorities was cross-referenced with interviews with professionals who work hands-on with the technology and could answer questions about labor, training, and the industry’s standardization.
Insights from the industry professional interviews focused on virtual production being a tool for production instead of a complete overhaul of traditional production practices. One professional stated that “any production that has replaced positions with a virtual supplement is not going to be successful...this technology is a tool...It does not replace traditional post-VFX pipelines at all...it is another way to get a shot” ( personal communication, December 21, 2022). We spoke with the VFX Supervisor who worked on the first season of The Mandalorian (2019) that revolutionized this technology. He agreed with this outlook, a sentiment reflected in our survey results as well: “It is a spectacular tool, but it’s only a tool...It is one tool that you can use to help tell a story” (personal communication, January 1, 2023).
We spoke to a DP who worked with LED screens on HBOMax’s Our Flag Means Death (2022). Their biggest surprise was “how little [his] process changed,” adding that “it’s opened up more possibilities than problems'' and has been brought up in every pre-production meeting “where that just wouldn’t have been the case even two years ago” (personal communication, January 13, 2023).
Given that many interviewees mentioned a lack of technicians and training as one of the major drawbacks of LED screen expansion, we spoke to virtual production educators about this issue. Training of virtual production technicians is “happening very quickly, but the demand is far greater than that” (personal communication, December 1, 2022). We spoke with a Senior Director of Film Technologies, and a Supervising Technical Director, both at New York University’s Tisch School of the Arts. Both are working to spearhead a virtual production program, one of many university-level virtual production programs being started across the country as the industry grows. They note that smaller companies offer to train technicians but, like many of our interviewees, agree that most technicians receive training on set and that standardizing training is a key to standardization of the virtual production process as a whole. They incorporated a virtual production program into the NYU curriculum because it was obvious; given the need, “it wasn’t brain surgery to say ‘Hey, this is important’” (personal communication, February 23, 2023).
Like our survey respondents, many interviewees agree that cost and amortization over time are two of the more prominent concerns impacting LED screen expansion. As professionals consider the need to invest in new technology, many worry that the investment will require upgrades sooner rather than later. Liebman stated they are “concerned it will be subpar technology by the time it is paid off” (personal communication, December 1, 2022). Through our interviews, we were able to confirm that a single day of shooting with an LED screen, including staffing, costs an average of $100,000 (personal communication, March 1, 2023), with many agreeing that once the technology is “at a more favorable price” and more “streamlined to execute,” uses for LED screens will expand far beyond just blockbuster movies and TV shows.
Regarding the issue of green screens, the responses were divided. Some respondents said there will always be space for green screens with creatives who need time to figure out the assets in post-production or productions requiring portability that LED screens cannot provide (personal communication, February 23, 2023). Others, like a SPE Chief Technology Officer, argue that green screen obsolescence is on the horizon, citing an LED screen’s capability of displaying a pure, flat green color which ultimately functions like a traditional green screen. This happens if technology glitches, thus rendering the need for both unnecessary ( personal communication, March 1, 2023).
What some considered a straightforward yes or no question became an industry-wide exploration of the tools creatives use to tell their stories. Not only did this imply that green screens will still have a place in the industry, but it also echoed a sentiment that our research respondents repeatedly voiced. The need to view LED screens as a new creative tool and not a complete upending of the production industry was repeated throughout interviews, emphasizing that the creatives and the story come first. As one professional shared, “The technology is really important, and the history of the technology is important, and the future of the technology is important, but the story is consistent” (personal communication, February 23, 2023).
RECOMMENDATION
With virtual production anticipating a 300% growth in market share by 2028 (Fortune Business Insights, 2023), studios must figure out how to approach the high-speed proliferation of LED screens without falling behind. The expansion of LED screens is not one to fear, though. Instead, many creatives choose to see it as an exhilarating time to tell new stories once considered impossible. Our survey findings and interviews highlighted consistent concerns related to virtual production: lack of trained labor, cost and amortization, and adaptability. Virtual production is still not standardized in an industry where standardization is the norm. Bill Baggelaar of SPE reiterates the idea of viewing LED technology as a singular tool.
BUILDING A TRAINING PROGRAM
Another concern we often heard was related to the demand for trained technicians in the virtual production industry. Universities are incorporating virtual production courses into their curriculums, but not at the rate studios need to keep up with the volume of productions using this technology. We recommend partnering with an organization like the American Film Institute (AFI), which is “dedicated to creating national education initiatives that inspire artists” (About AFI, 2023). A partnership like this will also guarantee a wider reach to more potential filmmakers, encouraging accessibility of virtual production training.
PRIORITIZING VIRTUAL PRODUCTION IN STUDIO SLATE
An essential part of amortizing LED screens' cost is ensuring they get used. Virtual production facilities around the world are being shut down because they cannot be monetized quickly enough, something ILM experienced with the Sydney-based stage they built for Thor: Love and Thunder (2022) (SB, personal communication, February 23, 2023). One way to prevent this is to rent out stages to other companies, and another is to encourage creatives to utilize the technology in their projects. “As long as [the LED screen] is meeting those needs” of the productions that are coming in, “you’re going to get much better value out of it” (personal communication, March 1, 2023).
As the survey responses showed, the possibility of lowered CO2 emissions is one of the primary factors determining whether a creative uses LED screens. However, the survey also revealed that a majority of industry professionals are unaware of the extent to which LED screens lower carbon dioxide output. Sustainability is clearly a high priority, especially for younger filmmakers. Quantifying carbon footprint offsets and their impact on lowering budgets, combined with the amplification of this sustainable benefit, could encourage more filmmakers to utilize LED screens for their projects.
“There’s [sic] many gains from” using LED screens from an artistic perspective, according to one of the interviewees, and communicating these to creatives who might be concerned about artistic limitations could help encourage them to try it. The LED screens’ interactive lighting is a game-changer for filming and is something a filmmaker cannot get with green screens. Another professional also praised the ability to shoot night scenes during the day, eliminating the need for exhausting late-night shoots (personal communication, January 13, 2023). For still-apprehensive creators, these opportunities could be the key selling point for LED screens.
As filmmakers learn more about the technology, consideration of LED screens can also be implemented in the development process. Story is still of utmost importance to a filmmaker, but if they were more aware of the possibilities an LED screen affords, they could incorporate it into the original storytelling and push the creative opportunities even further. One of the creators of the first season of The Mandalorian, knowing that he and Jon Favreau wanted to utilize and test this technology, encourages this sentiment as well: “If you start to then develop and design your shows, your environments, your scenes around what you know is going to work, then that's where you're going to have the most amount of success” (personal communication, January 1, 2023).
One of the wishes expressed in the survey was for more portability in LED screens, allowing them to be adapted for multiple projects. The response specifically cites MagicBox, a company that offers a mobile LED volume studio, as a great example of this. Built-in a trailer rig, MagicBox has a 17’x17’x 9’ volume that can be driven anywhere in the country (Kadner, 2022) (Figure C3 in the Appendix). ILM has begun a similar process after tearing down its Sydney-based volume after just one project. “Pop-up” volumes allow for portability, meaning that an LED screen can be combined with a physical location or that a project does not have to be limited to a specific Space. Building on our suggestion for improving the LED technology, we recommend greater exploration into crafting a more versatile and adaptable LED screen system
However, it is important to note that creating a one-size-fits-all LED screen is not necessarily ideal. As one professional said, filmmakers have to find “the right tool for the job.” There might be instances where the current Sony screen is not the tool needed. Instead, having a variety of screens on hand, each with a different purpose, can bridge the gap of being the “right tool” for any project. What might be suitable for Bullet Train (2022) might not be right for a smaller commercial or a live event.
CREATING A VIRTUOUS CYCLE
LED screens have already disrupted the production industry and spurred entire university curriculums to try and catch up with its growth. In five years, we can expect the proliferation of LED volumes to continue as more projects incorporate screens in their creative processes. In 2023, the virtual production industry is expected to reach $2 billion in market size (Grand View, 2023). While studios are racing to keep up with the virtual production industry, it is crucial to keep in mind Cassidy Pearsall’s thoughts about industry leaders: a leader in this field will be defined by “what they can do with the technology,” not just the fact that they have it (personal communication, December 21, 2022).
For studios to capitalize on the growth of virtual production and the opportunities accompanying it, centering the filmmaker and adapting the technology based on the production’s need is key. By creating a standardized training program, studios have the opportunity to build a reliable pipeline of skilled technicians. By prioritizing virtual production in their slate, they guarantee more projects for those technicians. Adaptable screens ensure virtual production can be used in multiple cases, not just big-budget features. Each initiative feeds into the others, creating a solid virtual production cycle that strengthens a studio’s position as a market leader. This not only helps amortize the screens but also establishes a more flexible creative environment.
-
About AFI. American Film Institute. (n.d.). Retrieved May 2, 2023, from
https://www.afi.com/about-afi/#:~:text=The%20American%20Film%20Institute%20is,i
nspire%20artists%20and%20audiences%20alike
ASC Staff. (2021, December 17). Sony releases Crystal led C-series, B-series displays. The
American Society of Cinematographers (en-US). Retrieved January 29, 2023, from
https://theasc.com/articles/sony-releases-crystal-led-c-series-b-series-displays
Baker, L. (2016, August 9). Create the Look of Natural Light with LED Screens and Projection
Panels. The Beat: A Blog by PremiumBeat.
https://www.premiumbeat.com/blog/create-the-look-of-natural-light-with-led-screens-a
nd-projection-panels/
Desowitz, B. (2020, May 8). 'the mandalorian' leads the way: Real-time virtual production is
saving Hollywood during the lockdown. IndieWire. Retrieved January 28, 2023, from
https://www.indiewire.com/2020/05/the-mandalorian-real-time-virtual-production-savi
ng-hollywood-lockdown-vfx-1202230120/
Fortune Business Insights. (2023, January 16). Virtual production market size worth USD 7.19
billion by 2029: Report by Fortune Business Insights. GlobeNewswire News Room.
Retrieved April 27, 2023, from
https://www.globenewswire.com/en/news-release/2023/01/16/2589144/0/en/Virtual-Pro
duction-Market-Size-Worth-USD-7-19-Billion-by-2029-Report-by-Fortune-Business-In
Sights.html
Giardina, C. (2022, October 19). Too much volume? the Tech behind 'mandalorian' and 'house
of the dragon' faces growing pains. The Hollywood Reporter. Retrieved February 1, 2023, from
https://www.hollywoodreporter.com/business/digital/volume-house-of-the-dragon-stage
-mandalorian-1235244158/
Grand View Research. (2023). Virtual production market size, share & trends report, 2030.
Virtual Production Market Size, Share & Trends Report, 2030. Retrieved April 7, 2023,
from https://www.grandviewresearch.com/industry-analysis/virtual-production-market
Guagenti, C. (2023, April 27). USD to CAD: Convert United States dollar to Canadian dollar -
forbes advisor. Forbes. Retrieved April 26, 2023, from
https://www.forbes.com/advisor/money-transfer/currency-converter/usd-cad/
ILM. (2022). Stagecraft. Industrial Light Magic. Retrieved February 4, 2023, from
https://www.ilm.com/stagecraft/
Internet Archive. (2011, February 28). SONY COMPUTER ENTERTAINMENT
ANNOUNCES RELEASE of PHYREENGINE TM 3.0. Wayback Machine. Retrieved
April 5, 2023, from
https://web.archive.org/web/20120321154129/http://research.scee.net/files/press/2011-0
3-PhyreEngine.pdf.
Leane, R. (2020, August 24). What do actors need to know about virtual production? -
backstage. What Do Actors Need to Know About Virtual Production? Retrieved April
25, 2023, from
https://www.backstage.com/uk/magazine/article/actors-virtual-production-71540/
Kadner, N. (2022, November 29). Welcome to virtual production foundations - virtual
production foundations video tutorial: Linkedin learning, formerly Lynda.com.
LinkedIn. Retrieved February 5, 2023, from
https://www.linkedin.com/learning/virtual-production-foundations/welcome-to-virtual-
production-foundations?autoplay=true
McKenzie, T. (2023, March 23). Adobe is partnering with Epic Games & Autodesk. 80lv.
Retrieved March 0, 2023, from
https://80.lv/articles/adobe-is-partnering-with-epic-games-autodesk-to-provide-game-cr
eators-with-powerful-tools/
Sakoui, A., & Lee, W. (2022, December 6). Amazon Studios unveils massive virtual production
stage, deepening ties to Culver City. Los Angeles Times. Retrieved February 4, 2023,
from
https://www.latimes.com/entertainment-arts/business/story/2022-12-06/amazon-culver-
city-virtual-production-stage-candy-cane-lane-eddie-murphy
Schaefer, S. (2022, May 24). The LED Screens Used In Our Flag Means Death Were So
Realistic They Made The Crew Seasick. Slash Film.
https://www.slashfilm.com/873215/the-led-screens-used-in-our-flag-means-death-were-
so-realistic-they-made-the-crew-seasick/
Seymour, M. (2019, September 2). How virtual production worked on-set of the lion king.
fxguide. Retrieved January 28, 2023, from
https://www.fxguide.com/fxfeatured/how-virtual-production-worked-on-set-of-the-lion-
king/
Sony. (2022, October 25). Press releases. Sony Pictures Entertainment Announces The
Acquisition Of Pixomondo, An Oscar And Emmy® Award Winning Virtual
Production, Visualization And VFX Company | Sony Pictures Entertainment. Retrieved
February 4, 2023, from https://www.sonypictures.com/corp/press_releases/2022/1031
Sony. (2023). Divisions. DIVISIONS | Sony Pictures Entertainment. Retrieved January 28,
2023, from https://www.sonypictures.com/corp/divisions.html
Sustainable Production Alliance. (2021, March). CARBON EMISSIONS OF FILM AND
TELEVISION PRODUCTION. Retrieved April 7, 2023, from
https://www.greenproductionguide.com/wp-content/uploads/2021/04/SPA-Carbon-Emi
ssions-Report.pdf
Sustainable Production Alliance. (2022, May). SPA SOUNDSTAGE FACILITY SURVEY
KEY TAKEAWAYS. Retrieved April 7, 2023, from
https://www.greenproductionguide.com/wp-content/uploads/2022/08/SPA-Report_05.2
5_FINAL_with_infographic.pdf
Tangcay, J. (2022, May 10). How virtual production is helping to cut costs and reduce carbon
footprint. Variety. Retrieved April 8, 2023, from
https://variety.com/2022/artisans/news/virtual-production-small-budget-1235236717/amp/
Tangcay, J. (2022, October 25). Sony Pictures Entertainment acquires VFX company
Pixomondo (exclusive). Variety. Retrieved April 6, 2023, from
https://variety.com/2022/artisans/news/sony-pictures-entertainment-acquires-vfx-compa
ny-pixomondo-1235413544/
Visual Effects Society. (2023, January 25). The virtual production glossary. The Virtual
Production Glossary. Retrieved February 4, 2023, from
https://www.vpglossary.com/#a-z-listing-letter-G-1