Written by Lauren Miller
Introduction
The combined budget of Peter Jackson’s iconic adaptation of J.R.R. Tolkien’s The Lord of the Rings trilogy was just under $300 million. Compared to the increasingly gargantuan budgets of contemporary stand-alone action or fantasy films, Jackson was working with a relatively modest financial outlay to make Tolkien’s Middle Earth, equipped with a star-studded ensemble cast, intricately designed creatures, and iconic, long-form battle sequences to come to life. This financial pressure hit Jackson the hardest while filming the Battle of Helm’s Deep for the franchise’s second installment: The Two Towers.
In the novels, the iconic Battle of Helms Deep sees Saruman combat the warriors of Rohan under the leadership of King Theoden Ednew in fight for the freedom of Middle Earth and control of the One Ring. Filming the battle was not an easy task, all in all, the battle took four months to film, with the majority of shots captured in the dead of night through the cusp of dawn in artificially simulated torrential downpours. “It was a nightmare,” candidly reported Legolas actor Orlando Bloom of the experience. To Peter Jackson, however, the grueling months took on an entirely new level of tension. “We all felt a bit under siege,” Jackson recalled; pressure from producers at New Line Cinema to keep the film under budget rattled head to head with the creative team’s commitment to appropriately capturing Tolkien’s pivotal battle. On top of this human labor, technology added critical elements of the ultimate final 40-minute cut of the Battle of Helm’s Deep with its ever-staggering scale were added to the film following the cast and crew’s grueling nighttime shoots in New Zealand’s North Island.
Scaling The Lord of the Rings Battle Sequences
Massive Software was developed by Founder and CEO Stephen Regelous in conjunction with New Zealand’s Weta Digital studio explicitly for use in Peter Jackson’s The Lord of the Rings film trilogy. At its core, Massive used artificially intelligent autonomous agents to generate animation. In layman's terms, software engineers train artificially intelligent systems on the movements and behaviors of agents (see: orcs, elves, Uruk-hai, etc.) in order to develop realistic hordes of animated characters in visual entertainment. As put by CNET writer Erin Carson, through its development specifically for The Lord of the Rings, Massive Software was able “to create computer generated armies using artificial intelligence” that were uniquely able to “simulate realistic battles on a tremendous scale.” Since its inception, Massive has received several Academy Awards for Scientific and Engineering Achievement and positions itself as the “leading software for crowd related visual effects and autonomous character animation.” The integration of the early AI capabilities contained within Massive Software alongside the unforgiving work of stuntmen, film crews, and actors were what made the battle between tens of thousands of fantasy creatures in Middle Earth jointly realistic in scale, financially attainable, and able to withstand the test of time.
The (Fuzzy) Logic Behind Massive Software and AI Animation
First implemented with The Lord of the Rings trilogy, Massive Software posits that the maintained believability of their AI-generated animation relies largely on the concept of “fuzzy logic.” Derived from mathematical theory and research, in the context of Massive, fuzzy logic provides a “way to create naturalistic and subtle behavior” that human viewers will interpret as realistic and powerful, even if there is no precise or logical information provided regarding its binary truth or reality. Fuzzy logic and the mathematical fuzzy set theory rose in popularity in the 1970s and 1980s particularly in Eastern Europe and the Soviet Union. The concept challenges classical, “Aristotelian, bivalent, [Cartesian, or Boolean] logic” that decision making and perception rely on “defined [boundaries] between two classes.” Fuzzy logic invites–as researcher Lotfi A. Zadeh puts it– “a progression from binarization to graduation, from binarism to pluralism, from black and white to shades of gray.” This shift calls on the human cognitive ability to understand classes of objects based on classes that may have unsharp (or, fuzzy) boundaries “with degrees of membership.” Fuzzy logic, then, allows the swarm of AI-generated orcs in The Battle of Helm’s Deep to move from what would have been considered uncanny or deeply unrealistic under traditional binary Cartesian logic to something just “sharp” enough to be cognitively classified as realistic or believable under fuzzy logic.
Profiting off of the implications of fuzzy logic on the human brain and AI programming, Massive Software has successfully been able to follow in a long line of trailblazing visual and special effects technologies in popular film and media. The usage, development, and implementation of computer based graphics in film has never been–and will never be–confined or defined by just one genre. However, it is insightful to observe how developments of the past have influenced Massive’s own technologies and its influence, predominantly in fantasy genres. To briefly summarize: The earliest influences on visual effects as we know them today are attributed to the first usage of 2D computer animation in the 1973 film Westworld, as well as the first use of 3D computer animation in its 1976 sequel Futureworld. Star Wars: A New Hope ushered in an era of 3D widescreen graphics in science fiction, leading to the unveiling of the first “fully 3-D digital (or CGI) … photorealistic animated character in a full-length feature film” in the 1985 film Young Sherlock Holmes. The potential for computer generated water effects were made public with James Cameron’s 1989 The Abyss, physically textured graphics were prominent on the iconic dinosaurs in 1993’s Jurassic Park, and–of course– Pixar’s Toy Story was released in 1995 as the first fully computer graphics generated feature film. Large-scale CGI battle sequences featured in films such as Starship Troopers in 1997, as well as the simultaneous release of Andy Serkis’ motion-capture-created Gollum in The Lord of the Rings trilogy, directly paved the way for the development of Massive’s technological intersection of computer generated effects with early artificial intelligence.
Drawing influence from the visual developments of the past, “Massive … helped elevate what audiences now expect from onscreen battles.” Notably, however, over 20 years since the release of The Two Towers, the Battle of Helm’s Deep looks just as realistic as it did in original screenings. Once again, this opportunity for longevity ties back to fuzzy logic. In addition to the mere perception of AI-generated agents as realistic by the cognitive system of a human viewer, fuzzy logic allows for diversity and unpredictability in how agents will interact with each other–adding an entirely new level of dimension to the reliability or believability of the software. To illustrate, for every individual agent in Massive Software–say, for the purpose of discussion an orc–fuzzy logic states that there is no “true” or “false” way that they can interact with an adversary. Rather, the orc is essentially free to do any number of things “in between” the “true” (plausible) or “false” (implausible) nature of its programming set by film creatives and software engineers. When this range of options for a single orc is multiplied by the number of separate agents and logic rules guiding their interactions, users of Massive Software are left with multitudes of unique actions, interactions, and outcomes for each individual agent. Flying over Helm’s Deep, viewers do not see thousands of identical orcs simultaneously raising their hammer to a cowering Man of Rohan, rather they perceive a network of complex, living entities exhibiting unique interactions with adversaries, sounds, and landscapes on the battlefield.
This ability to program, in essence, freedom or will into Massive Software’s agents through the laws of fuzzy logic not only have allowed the software to stand the test of time, but have allowed it to expand exponentially since the film’s initial release. Massive’s “Ready-to-Run” agents “can be dropped into shots to create 3D character-based crows of extras, allowing users to significantly speed up their workflow.” These agents are regularly updated or launched to expand Massive’s network of AI generated possibilities. For instance, in 2017 the company announced their “Horse and Rider” agent. The technology allowed for the programming of 100 individual movements and actions on a large scale for AI-driven horse and rider agents. All generated agents are pre-built with a “skeleton, motions, brains, geometry, clot, textures, shaders, controls” and “dynamic hair simulation” that may be further customized in order to create the unique physical appearance or patterns of movement desired by users. Looking to the future of Massive Software and AI-generated animation in fantasy film, Stephen Regelous noted that the technology prevents potentially stressful and dangerous conditions for human and horse actors alike, as well as increases the realistic scope of shots or action sequences that less financially-profitable studios can afford to have in their works.
Massive Software’s Role in Contemporary Fantasy Entertainment
By mere nature of scale and otherworldliness, fantasy entertainment is ambitious to develop. As discussed extensively above, however, programs such as Massive Software have significantly eased some of the production pressure in the creation of these worlds over the past two decades.
Massive Software was central to the development of the epic battles in HBO’s adaptation of George R.R. Martin’s fantasy epic Game of Thrones. Having been implemented sporadically throughout the show’s eight-year run in various capacities to capture the vast expanses of Westeros and the lands beyond the Narrow Sea, the integration of Massive culminated with its deployment in the Battle of Winterfell: the longest battle sequence in film or TV history to date. In the show’s final (read: highly contested) season, the Battle of Winterfell was the clash between Westerosi and Dothraki armies and the undead “white walkers,” wights, and Night King from beyond the wall. The battle presented an interesting challenge for Game of Thrones’ visual effects supervisors; distinct movement patterns of creatures slowly developed over the course of seven previous seasons (upwards of 50 hours of content) had to be translated to logical movements in unprecedented swarms while simultaneously interacting with distinctive landscapes and adversaries. In other words, the scale of individual logic patterns that had to be established for each individual creature was–if you will–massive.
As has been established, however, Massive Software operates at the level of individual, artificially intelligent agents. Take, for instance, a wight storming the castle in the Battle of Winterfell. In Game of Thrones, wights are somewhat more complex than the average zombie as they possess an evident understanding of their surroundings and a loosely logical decision-making process when faced with changes in environment. As visual effects supervisor Martin Hill explained of wight consciousness: “they’re all hungry to get inside [the walls of Winterfell], but not t the extent where they’ll pull another wight down to get in front of him … they’re not trying to get one-up over the wight next to them.” Using the advancements to Massive that had deployed since its initial launch with The Lord of the Rings, each wight was provided a field of view, which allowed it to react realistically to its surroundings. Each of the artificially intelligent wight’s perceptions of their point of view were then fed into a motion capture library, which “triggers different motion cycles or different parts of the capture library” that allow the agents to freely and logically scramble through the environment. Teams of stunt-people, physical performers, and motion capture artists were combined with a database of pre-existing wight movement in order to “enhance” the motion of the artificially intelligent agents. In the end, the network of motion capture possibilities open to the agent was “at the point of not really being countable” and 30,000 of these environmentally conscious, intelligent wights ultimately descended upon the walls of Winterfell.
Looking Forward: Competitors, Markets, and Relevancy
Massive Software has unquestionably defined what modern viewers conceptualize as large-scale fantasy or action battles and crowds. The technology’s interface and ever-adaptable usage of fuzzy logic in programming and user perception have made its products stand the test of time–with moments appearing just as realistic today as they did at their time of initial release. However, it cannot be lost that Massive Software is over two decades old. As technologies continue to advance and the novelty of AI technologies have begun to wear down, Massive alone cannot power the fantasy and action sequences in the film and television industry. This does not mean, of course, that the popular usage of Massive Software is fading (see: The Suicide Squad, Avengers: Endgame, Edge of Tomorrow, Doctor Who, etc). Rather, the future of the technology lies in its ability to be implemented with its competitors.
For instance, the pivotal final Battle of Ta Lo in Marvel’s 2021 epic Shang-Chi and the Legend of the Ten Rings took a team of 250 individuals, all working predominantly with a blend of Weta Digital’s Massive Software, Houdini, and Koru Animation to create the terrifying creatures and armies backing Shang-Chi and his titular rings. In particular, Ta Lo’s “soul sucking demons” were developed with a combination of Massive crowd and Houdini simulations, vignettes, and keyframe animation.” Of particular note to the present discussion, these demons would not have been possible to create without the power of both of the niches Massive and Houdini fill: Massive for the development of artificially intelligent agents and Houdini for creating higher realistic detail on these agents through “modeling, rigging, animation, lighting, and more.”
Where Massive was once, in essence, the end-all-be-all of crowd generation, new technologies have allowed for the supplementation or augmentation of these crowds to make even more viscerally impactful and realistic worlds. From Aragorn throwing open the doors at Helm’s Deep backed by an endless sea of adversaries through Arya Stark weaving through the hordes of undead, Massive Software has developed some of the most iconic fantasy action sequences in popular entertainment to date, and will likely continue to well into the future.
-
“About Us.” Massive. Accessed March 14, 2023. https://www.massivesoftware.com/about.html.
Carson, Erin. “How ‘Lord of the Rings’ Used AI to Change Big-Screen Battles Forever.” CNET. September 4, 2022. https://www.cnet.com/culture/entertainment/features/how-lord-of-the-rings-used-ai-to-change-big-screen-battles-forever/.
Combrinck, Tanya. “How Weta Created Astounding VFX for Marvel’s Shang-Chi.” News. VFX Wire. October 18, 2021. https://www.vfxwire.com/how-weta-created-astounding-vfx-for-marvels-shang-chi/.
Dasiwick, Tyler. “15 Years Later, No-One’s Matched ‘LOTR’s Battle at Helm’s Deep.” Relevant. December 18, 2017. https://relevantmagazine.com/culture/movies/15-years-later-no-ones-matched-lotrs-battle-at-helms-deep/.
“Features.” Products. Massive Software. Accessed March 16, 2023. https://www.massivesoftware.com/features.html.
“Film.” Industries. Massive Software. Accessed March 27, 2023. https://www.massivesoftware.com/film.html.
Gonzalez, Ronnie. “GoT Battle of Winterfell Breaks TV & Film Records.” 96.3 KKLZ. April 29, 2019. https://963kklz.com/2019/04/29/got-battle-of-winterfell-breaks-tv-film-records/.
Jevon, Lee. “We Did the Maths: Here’s How Long It’d Take to Watch All of Game of Thrones.” TV. Starte Facts. February 22, 2023. https://startefacts.com/news/we-did-the-maths-here-s-how-long-it-d-take-to-watch-all-of-game-of-thrones_a125#:~:text=Particularly%20as%20watching%20every%20episode,7
0%20hours%20and%2014%20minutes.
“Movie Budgets.” Research Tools. The Numbers. Accessed March 26, 2023. https://www.the-numbers.com/movie/budgets/all.
Mussi, J.L. “Houdini Review.” Photo and Design. PC. October 17, 2022. https://www.pcmag.com/reviews/houdini.
Otway, Jack. “All Lord of the Rings and Hobbit Movies, Ranked by Budget.” Movie Lists. Screen Rant. August 21, 2020. https://screenrant.com/all-lord-rings-hobbit-middle-earth-movies-ranked-budget-highest-lowest/#the-return-of-the-king-2003-94-000-000.
“Ready-to-Run Agents.” Products. Massive Software. Accessed March 26, 2023. https://www.massivesoftware.com/massiveagents.html.
Schaeffer, Sandy. “Peter Jackson Was Fighting His Own Battle During The Lord of the Rings’ Helm’s Deep Scene.” Slash Film. Updated July 28, 2022. https://www.slashfilm.com/943405/peter-jackson-was-fighting-his-own-battle-during-the-lord-of-the-rings-helms-deep-scene/.
Stanley, Anya. “Starship Troopers is Still a Point of Pride for ILM Effects Legend Phil Tippett.” Slash Film. December 3, 2022. https://www.slashfilm.com/1118527/starship-troopers-is-still-a-point-of-pride-for-ilm-effects-legend-phil-tippett/.
Tangcay, Jazz, Katie Reul, Julia MacCary, and Charna Flam. “Academy Awards to Honor Eight Scientific and Technical Achievements – Film News in Brief.” News. Variety. February 2, 2023. https://variety.com/2023/film/news/film-news-in-brief-jan-31-2-1235506426/.
“Television.” Industries. Massive Software. Accessed March 27, 2023. https://www.massivesoftware.com/television
Tibbs, Ross. “Timeline: A Brief History of CGI in the Movies.” Film. Far Out. February 28, 2023. https://faroutmagazine.co.uk/timeline-history-of-cgi-movies/.
Trenholm, Richard. “Westworld’s TV Twists Were Hidden in Futureworld All Along.” Entertainment. CNET. April 18, 2018. https://www.cnet.com/culture/entertainment/westworld-twists-were-hidden-in-futureworld-all-along-hbo/.
Williams, Hannah Shaw. “The Truly Epic Story of How the Battle of Helm’s Deep was Filmed.” Slash Film. Updated December 18, 2022. https://www.slashfilm.com/1125056/the-truly-epic-story-of-how-the-battle-of-helms-deep-was-filmed/.
Wolfe, Jennifer. “Massive Announces AI-Controlled Horse and Rider Agent.” News. Animation World Network. July 31, 2017. https://www.awn.com/news/massive-announces-ai-controlled-horse-rider-agent.
Zadeh, Lofti A. “Fuzzy Logic– A Personal Perspective.” Fuzzy Sets and Systems 281, no. 1 (May 21, 2015): 4 -20. https://cmu.primo.exlibrisgroup.com/discovery/fulldisplay?docid=cdi_gale_infotracacademiconefile_A520068181&context=PC&vid=01CMU_INST:01CMU&lang=en&search_scope=MyInst_and_CI&adaptor=Primo%20Central&tab=Everything&query=any,contains,fuzzy%20logic&facet=rtype,include,articles&mode=Basic&offset=0.
Zakarin, Jordan. “Artificial Intelligence-Powered Movie Characters Are On the Way.” Entertainment. Inverse. March 31, 2017. https://www.inverse.com/article/29770-ai-movie-characters-game-of-thrones-dragons-vfx.
Zakarin, Jordan. “‘Game of Thrones’ Battle of Winterfell Was Powered By Artificial Intelligence and Horse Treadmills.” Wire. SyFy. June 11, 2019. https://www.syfy.com/syfy-wire/game-of-thrones-battle-of-winterfell-was-powered-by-artificial-intelligence-and-horse.