by Tatiana Behar Russy
Technology has influenced the film industry since its inception, as advances in the film industry are always closely related to technological advances and how they can be applied in the filmmaking process. Technology has had a deep impact on revolutionizing every aspect of filmmaking, from pre-production to post-production, from digital cameras, editing software, sound and music, distribution, and exhibition, to special effects. Almost every aspect of filmmaking, if not all, has undergone major changes and has evolved over the last 50 years.
Traditional film production involves building physical sets, scouting locations, and coordinating large crews and equipment, all of which can be time-consuming and expensive. To overcome some of these "problems" or “needs” was born what we know today as Virtual Production.
Source: Frame.IO (https://blog.frame.io/2021/09/07/10-things-to-know-about-virtual-production/ )
Virtual Production and the film industry
Virtual production is a filmmaking method that involves creating digital sets and using them to shoot live-action scenes, combining virtual and physical worlds to create movies. This technique enables filmmakers to replace traditional physical sets and locations with virtual ones, which can be designed and modified in a much more flexible and cost-effective manner (Chanpun, 2023). In virtual production, filmmakers use visual effects technologies to create highly detailed and realistic digital environments, creating hyper-realistic environments.
The beginning of virtual production was back in 1930 rear projection techniques appear trying to recreate atmospheres with a projection, where the location and the actors were recorded at different times, as did Liliom, one of the first films to use this technique. Then, in 1968, Stanley Kubrick used front projection in the movie 2001: A Space Odyssey.
But, how does it work now? With the evolution of technology, nowadays actors perform against a LED wall -that is replacing the famous green screen- that displays the virtual environment, which is generated and manipulated in real-time by the production team. Do you notice how revolutionary this is? Basically, post-production is now pre-production, because all the visual effects are made even before shooting the scenes. This technique allows filmmakers to adjust lighting, camera angles, and other visual elements on the set and during the production, giving them more control over the creative process.
Virtual production has found applications in various movies, from high-budget sci-fi and fantasy movies to low-budget independent productions. Additionally, virtual production has also been utilized in other media forms, including video games and virtual reality experiences. As technology progresses, virtual production will become even more significant and impactful in the film industry.
What about Artificial Intelligence?
All of the above is possible thanks to Artificial Intelligence (AI).
AI plays an essential role in virtual film production by assisting in the creation of realistic and immersive digital environments (Hogg, 2021). Here are a few ways AI works in virtual film production:
AI is used to generate content, such as landscapes, architecture, and vegetation. This allows filmmakers to create digital environments quickly.
Machine learning is used to analyze and track the movement of objects and actors on a virtual set. This helps to create more realistic and natural-looking scenes by ensuring that objects and actors move and interact with each other realistically.
AI is used to capture the movements of actors and objects in real-time to record motion data, which can be used to animate digital characters and objects more accurately.
AI is used to generate complex special effects and lighting effects that would be challenging to create using traditional methods.
AI is used to render high-quality images and footage in real-time. This allows filmmakers to preview and adjust their digital environments and scenes quickly and easily.
By leveraging the power of AI, filmmakers can now design and construct digital sets that are incredibly detailed, convincing, and dynamic. AI is being used to enhance every stage of the filmmaking process. For instance, Filmmakers can employ AI tools to create 3D models of characters and props that can be used for virtual production. Furthermore, they can use machine learning algorithms to predict a movie's success or analyze feedback from the audience, enabling them to modify their productions to meet the preferences of their target audiences.
Benefits of Virtual Production
This technology offers significant cost and time savings, as filmmakers can avoid the expense of location shoots and create complex and elaborate sets that would be impossible to build in the real world. It also allows for more efficient and cost-effective filmmaking, as it eliminates the need for expensive location shoots and allows for quicker turnaround times (Kamat, 2022).
The more important benefits of film virtual production are:
Cost savings: Virtual production reduces the need for expensive physical sets, locations, and props, resulting in significant cost savings.
Flexibility: Filmmakers can create and modify virtual environments and scenes quickly and easily, providing them with greater creative control and flexibility.
Real-time visualization: Virtual production allows filmmakers to preview and adjust their scenes and environments in real time, enabling them to make informed decisions and save time during post-production.
Improved safety: Virtual production provides a safer alternative to traditional physical production, as it eliminates the need for actors and crew members to work in risky environments.
More realism: Virtual environments and characters can be made to look more realistic, resulting in more immersive and engaging films.
Increased efficiency: By using virtual production techniques, filmmakers can streamline their production processes, resulting in faster and more efficient productions.
Movies and series made with Virtual Production
Movies and TV shows like "The Lion King", "The Mandalorian", "Avatar", "The Jungle Book", "Ready Player One", and "Gravity" have used virtual production techniques. These productions used various tools and technologies, including motion capture and virtual environments, to create realistic and immersive worlds for the actors to perform in. These techniques have helped to push the boundaries of what is possible in filmmaking, and have resulted in visually stunning and memorable final products (An, 2022).
"The Lion King" (2019): This film used virtual production to create realistic, computer-generated environments and animals, which were based on extensive research and observation of real-life animals in the wild. The virtual production team used motion capture to capture the movements of the actors and then integrated these performances into the digital environments to create a seamless, photorealistic final product.
"The Mandalorian" (2019): The TV series was shot using a cutting-edge virtual production stage called "The Volume," which utilized LED screens and real-time rendering to create detailed and immersive environments for the actors to perform in. The virtual production team used motion capture and other techniques to integrate the live-action footage with the digital environments.
"Avatar" (2009): One of the first major movies to extensively use virtual production techniques, including motion capture and virtual environments. The virtual production created the film's lush, alien world of Pandora, which featured incredible details and realism.
"The Jungle Book" (2016): The movie was shot entirely on a virtual production stage, with the actors interacting with computer-generated environments and animals, integrating the actors' performances with the digital set.
"Ready Player One" (2018): In this film, they created a complex and immersive virtual world, based on the novel of the same name.
"Gravity" (2013): While not shot entirely with virtual production techniques, Gravity utilized extensive previsualization and digital compositing to create its realistic and breathtaking space scenes. The virtual production team worked closely with the director and cinematographer to plan out the shots and integrate the live-action scenes with the digital effects.
Conclusion
Overall, AI enables filmmakers to create more realistic, immersive, and cost-effective virtual environments that were previously impossible with traditional filmmaking. Furthermore, while the examples above come from mega-studios, AI tools are in the hands of independent filmmakers as well, particularly in post-production.
The future of virtual production in movies seems bright. Advancements in technology are likely to make virtual production more sophisticated and integrated with other filmmaking techniques. Real-time rendering, expansion into other genres, more accessible tools and technologies, and greater collaboration are some of the trends that will shape its future. As a result, virtual production will enable filmmakers to create more innovative and creative projects, while offering greater flexibility and efficiency in the production process.
-
An, D. (2022). Technology-driven Virtual Production. Revista FAMECOS. V 29. https://revistaseletronicas.pucrs.br/index.php/revistafamecos/article/view/43370/27562
Bickerton, J. (2022). Virtual production innovation series. Broadcast, Retrieved from https://www.proquest.com/trade-journals/virtual-production-innovation-series/docview/2638962072/se-2
Chanpun, P. (2023). Virtual production: Interactive and real-time technology for filmmakers. Humanities, Arts and Social Sciences Studies. https://so02.tci-thaijo.org/index.php/hasss/article/view/260250/175022
Fandor. (2019). Rear Projection: How It Works. https://youtu.be/dwe4Fan41Is
Hogg, T. (2021). Perspectives: Chasing the evolving industry norm for virtual production. VFX Voice. Issue Fall 2021. https://www.vfxvoice.com/perspectives-chasing-the-evolving-industry-norm-for-virtual-production/
Industrial Light & Magic. (2020). Virtual Production of “The Mandalorian” Season One. https://youtu.be/gUnxzVOs3rk
Kamat, S. (2022). Atmospheres of Change: Virtual Production. Massachusetts Institute of Technology (MIT). https://dspace.mit.edu/handle/1721.1/147235
VFX Guru. (2018). Gravity (2013) - MakingOf https://youtu.be/SAfmjQfJpeo
Vox. (2021). The technology that’s replacing the green screen. https://youtu.be/8yNkBic7GfI