Written by Emily White

Introduction

Few innovations represent the intersection of humanity and technology more famously than that of the robot. Like art, robots are the result of humanity’s urge to create something new in its own image – and, also like art, they have become an inescapable part of our world. In the 21st century, robots have surpassed the limitations which restricted them to industrial manufacturing and technological research, now occupying roles across all industries and disciplines. Incorporating robotics into the arts industry blends two very different expressions of human creativity: bringing exciting new technological innovations to the arts and giving robotics a touch of humanity.

This article will trace the evolution of robots from mythological and historical automata to science fiction characters, industrial innovations, cutting-edge research subjects, and their status as specialized workers. This will be followed by an outline of the characteristics of today’s robots, particularly their three distinct roles in art (intermediary, tool, and artist) through case studies and examples.

Rooted in Fiction: Conceptual Origins of the Robot

Robots occupy a sizable chunk of the modern imagination thanks to a rich literary tradition of science fiction dating back to the 19th century, rooted in the supernaturally-created and -animated humanoids of many ancient folklore traditions. The first mechanical android resembling what we now think of as the archetypal robot made its science fiction debut in Edward S. Ellis’ 1868 novel The Steam Man of the Prairies.  However, the word “robot” itself was not coined until 1920 by Czech writer Karel Capek.  It derives from the Czech word “robotnik,” or “forced worker.” Capek created this word in order to identify a fictional group of manufactured, sentient workers who overthrow their oppressive human overlords in his play RUR (Rossum’s Universal Robots). Unlike the robots of real life (and more recent works of science fiction), Capek’s robots were created from flesh and blood, much like the famous man-made humanoid from Mary Shelley’s Frankenstein, predating RUR by fifty years. Science fiction author Isaac Asimov further developed the concept of robots by using the term to refer to mechanical humanoids much like Ellis’ Steam Man – whose character embodied the values that Asimov would later distill into his Three Laws of Robotics. Asimov optimistically reimagined robots as “helpful servants of man,” his famed Three Laws of Robotics enshrining the view of robots in the public imagination for decades afterwards.

Brave New World: History of Modern Robotics

Robots are not only a fictional concept, but an increasingly integral part of our daily lives. Outside science fiction, a robot is defined as “a machine capable of carrying out a complex series of actions automatically, especially one programmable by a computer.” While automata date back to the Middle Ages in Europe and the Middle East, it was Victorian scientists Charles Babbage and Ada Lovelace who laid the groundwork for mechanical computations and computer creativity.  Despite the early conception of this idea, it was not until the 1950s that the first industrial robot, George Devol’s Unimate robotic arm, began its work on the General Motors assembly line.  This opened the door for automakers around the world to invest in articulated manufacturing robots and develop the automated welding lines that would eventually take over the global auto industry.

The second half of the 20th century saw innovations in improving industrial robots’ existing functions, exploring imaginative new abilities, such as mobility, artificial intelligence, computer vision, autonomous control, and developing humanoid design. In the 21st century, rapid technological advancements, and a turn-of-the-millennium limit-pushing futurist mindset, created an explosion of new developments in the field of robotics such that today, there is a specialized robot for practically everything. Nonfictional robots, like their fictional counterparts, are mostly created to work, doing so in a wide variety of roles.  These may span from the utilitarian, such as manufacturing, disaster relief, medical care, and military applications, to the interactive or even imaginative, such as education, telepresence, companionship, and entertainment. It is in this latter category of roles that robots have slowly gained a foothold in the arts and entertainment industries, where they assist, engage, and interact with humans.

Structure Determines Function: Core Commonalities and Divergent Designs of Specialized Robots

Despite the differences among today’s diverse and varied selection of highly specialized robots, they all have three things in common: they are constructed of interrelated mechanical components; they require electrical power sources and control components; and they operate through coding commands of varying complexities. Each possess different characteristics determined by the purpose for which they were created. For example, a robot manufacturing car parts at an auto plant must perform a set of precise and complex movements, but it does not need to communicate with humans, “think” independently, or move.  Thus, it takes the form of a stationary articulated robot configured like an arm with multiple twisting joints, pre-programmed to perform its specific task with no other function. Conversely, a robotic Mars rover needs to autonomously move around the planet’s surface for terrain exploration and sample collection, so it is equipped with a camera, sensors, wheels and a motor, various grasping arms and instruments, a processor that enables it to make decisions, as well as equipment for transmitting data back to Earth. A robot intended for human interaction, such as Sophia of Hanson Robotics, needs to be able to communicate verbally with humans.  Therefore, these types of robots require an interface with which a human can feel comfortable conversing, such as a hyperrealistic humanoid face, and would ideally be equipped with some form of artificial intelligence so that it can follow a human’s cues and converse without direct human control. A robot’s specialized hardware determines its capabilities and the roles for which it is equipped.

The Many Roles of Robots:

THE ROLE OF ROBOTS IN THE ARTS

Robots as Intermediaries

In 2020, the Covid-19 pandemic safety measures left museums and galleries closed and would-be visitors isolating at home. Face-to-face communication with friends, family, classmates, and coworkers was only possible via video chat.  Almost overnight, virtual became the norm.  While art museums and galleries followed suit through offering virtual gallery talks and highlighting online audience experiences, they cannot compare to in-person activities.

Initial shutdowns in March 2020 hindered arts institutions greatly, but by April, several enterprising museums and galleries had found a temporary solution: they could use tele-operated ambulatory telepresence robots to virtually bring visitors back into their venues. The robots were equipped with a view screen and a camera at eye level, enabling them to act as intermediaries between the visitor and the museum. Hastings Contemporary in London adopted this practice early in the pandemic by borrowing a $4,000 Double Robotics bot from the Bristol Robotics Laboratory just before the UK shutdown. Virtual visitors tele-operated the ambulatory robot through their web browsers, which allowed them to see into the space through the robot’s webcam, watch where they were driving, and explore the gallery’s collection as if they were on-site. The tele-operation feature gave visitors control over their virtual experience, and the robot’s ambulatory nature added the element of physical movement through space that stationary video conferencing lacks, erasing the barrier of the screen and creating a feeling of physical presence in the gallery.

Hastings Contemporary was not the only gallery to take advantage of this technology: Avant Gallery in New York City used Ohmni telepresence robots as intermediaries to virtually welcome collectors to the gallery, resulting in a continued flow of sales and commissions. This experience was enhanced by the Ohmni’s high-quality camera secured on a swiveling neck that allows collectors to see through the robot’s video screen with a full 360-degree view, enabling them to speak with gallery staff face-to-face.

Telepresence robots of the tele-operable and ambulatory variety are not new inventions, nor are they new in the museum world.  In 2014, the Tate Britain collaborated with The Workers, a digital design studio, to offer five nights of after-hours telepresence robot tours to internet users interested in visiting the Tate from home. The event, called “After Dark,” won the Tate 2014 IK Prize, an award given in recognition of projects that harness digital technology for arts accessibility.

Case Study: Beam Telepresence Robot

In October 2015, the Van Abbe Museum in Eindhoven, Netherlands began to offer art lovers who had difficulty visiting the museum in person due to physical disabilities the opportunity to conduct virtual visits via Beam Telepresence Robots as part of their “Special Guests” accessibility program. Similar to telepresence robots, Beam telepresence robots are ambulatory and tele-operable, making them ideal for the intermediary role of connecting virtual visitors to museum spaces.

The museum stressed the importance of empowering virtual visitors to control their museum experience by driving their own Beam robot. Beam telepresence robots are equipped with a screen and camera placed approximately at eye level to facilitate conversation.  This feature enabled virtual visitors to interact easily with the guide who accompanies each Beam robot through the museum. In addition, the novelty factor of the telepresence robot tends to attract attention and interaction from in-person museum visitors, who often strike up conversations with virtual visitors.  These conversations deepen the interactivity and further personalizes the museum experience of virtual visitors. Although the Tate Britain hosted its “After Dark” five-night telepresence robot program one year before the Van Abbe Museum launched Beam telepresence visits, the Van Abbe Museum was the first in Europe to make robotic telepresence visits a permanent part of its accessibility program, bringing art and museums back into the lives of physically disabled art lovers.

While the Hastings Contemporary telepresence robot could be controlled through a web browser, the Beam telepresence robot requires a software download. A team of reviewers from the Invisible Studio design agency found in 2018 that the software was easy to use, that tele-operating the robot provides a realistic sense of the physical space, that the Beam robot works very well for speaking with the museum guide and in-person visitors, and that the experience provides a good museum ambiance.  However, the camera quality was too low and the angle too wide to provide a good view of the art, which is unsurprising, as Beam is primarily a teleconferencing robot.

Robots as Tools

A good programmer can make a robot do almost anything – acting and singing are no exception. Pre-programmable robots with semi-humanoid or entirely humanoid designs are ideal for playwrights, directors, and other theater and opera professionals to use as tools to express their theatrical visions.

In 2013, Japan Society’s Robot Theatre Project began to present plays written by playwright Oriza Hirata in which human actors perform alongside pre-programmed androids. Sayonara, a play where a human actress and a hyper-realistic android named Geminoid F recited poetry back and forth, involved careful timing: Geminoid F was pre-programmed to deliver its lines at certain times, and actress Bryerly Long had to deliver her lines in those timing gaps. The restrained style of Hirata adapts well to robotic performers, since it relies on strictly timed delivery and subdued emotion, intending to “leave the role of emotional resonance with the audience.” Since Geminoid F is tied to a pre-programmed script, it hinders the robot’s ability to create energy from interacting with other performers.  This is only a disadvantage for productions in which genuine emotion from the performers is more important than control, precision, and exact execution of the playwright’s vision. For Hirata, Geminoid F is the perfect medium of expression.  In fact, many audience members that night were unaware that the hyperrealistic android was not human.

Death and the Powers, a 2011 robotic opera collaboration between human performers and geometric singing OperaBots, presents a different theater-robot experience. Composed by Tod Machover, director of MIT Media Lab’s Opera of the Future Group, “Death and the Powers” is the tale of a billionaire who, fearing death, uploads his consciousness to the cloud and leaves his body behind. The opera’s themes of technology are underscored by the OperaBots chorus, which are pre-programmed using a specialized robot choreography software (that can be tele-operated by puppeteers using video game controllers if something goes wrong). The OperaBots have flat triangular “heads” which move expressively as they sing the chorus.  Due to their flexibility between pre-programming and tele-operation, they are useful tools for directors with less rigid artistry.

Case Study: RoboThespian

RoboThespian is an acting humanoid robot developed by Engineered Arts, a theatre technology company. Equipped with interchangeable facial features, hair pieces, other accessories, and RGB LED lights that illuminate the robot’s body from within, RoboThespian was developed expressly for use as a means of executing the exact vision of a director or performer.

RoboThespian is capable of expressive vocalization (both sung and spoken) as well as gestures and postures.  However, it is not ambulatory without the application of Smooth Mover, another Engineered Arts product that uses tracks to transport RoboThespians around the stage. Theater professionals can use Virtual Robot software, also created by Engineered Arts, to easily pre-program RoboThespian’s performances down to every word, gesture, sound, and lighting design  cue.  An entire theatrical performance with multiple RoboThespian actors and complex lighting and sound design can be automatically executed at the press of a button. Their click-and-drag interface and timeline-based performance structure requires no coding knowledge to program a RoboThespian performance.  Additionally, its compatibility with third-party devices gives directors the flexibility to use their own lights, speakers, smoke machines, and any other special effects technology.

RoboThespian’s apparent robotic appearance may turn off theatergoers who prefer to watch human actors.  Regardless, there has been success at the Copernicus Science Center’s theater, which runs an average of eight RoboThespian performances daily,  compared to the frequency of traditional theater productions, which is one or two daily performances. RoboThespian is the ideal robotic tool for executing a fun, low-maintenance, innovative, and futuristic theater performance for those with little programming and directing experience.

Robots as Artists

Computer creativity and artificial intelligence are popular topics in technology, underscoring the rise in artificially intelligent robotic “artists” in recent years. Robotics innovators are putting Ada Lovelace’s artificial creativity theories to the test by using neural networks, deep learning, and other techniques in attempts to make a computer produce organic ideas. Many of these new AI robots are robot painters, but their designs vary from a simple articulated arm robot, like those used in manufacturing, to a highly realistic humanoid whose appearance has more to do with human interaction than with painting ability. Artificially intelligent robot musicians also exist, including an android conductor that hums along as it directs human orchestras with realistic gestures. These robot-artists are equipped with different forms of artificial intelligence so that they may autonomously create.  However, none are ambulatory, as their art forms do not require movement around a space.

A new painter debuted at South By SouthWest this spring – a robot created as part of the AI Robot Project from the University of Tokyo in Japan. Unlike most AI painting robots, which use generative algorithms to create artworks, the University of Tokyo’s robot uses a rules-based neural network as a source of guidance, which taught the robot artistic theories and concepts, akin to sending the robot to art school. The project’s goal is to “develop an independent creative entity to augment human creativity,” according to Jiamin Pan, the project leader.  Its researchers hope to achieve this through giving the robot concept images from which to draw, as well as challenging instructions such as limiting the number of brushstrokes used in order to creatively train it to adapt to limitations.

Ai-Da, a humanoid AI painting robot created at Oxford University, shares an art form with the University of Tokyo’s robot, but that is where the similarity ends. Ai-Da is frequently referred to with she/her pronouns and was designed to look like a human woman. Named after Ada Lovelace, Ai-Da can converse with humans, and is marketed as “the world’s first robot artist,” having her own solo exhibition at Oxford. Her artistic method is more complex than that of most painting robots: sensors, AI, and facial recognition technology allow her to use computer vision to “see” her subject.  With some help from algorithms, she uses her robotic arms to create a sketch, which researchers at Oxford then plot on a graph and process with a neural network that analyzes the design, creating a prismatic pattern based on the drawing.  This pattern is printed on canvas, where a human artist will add finishing touches by painting over parts of it. There may be some discourse in asserting if Ai-Da is a real artist, since she only creates sketches and then others execute paintings based on those sketches, but this argument is flawed, as this is the technique that some human artists employ, such as Sol Lewitt. Ai-Da is certainly an ambitious artificial intelligence project whose goal is to work towards not only computer creativity and computer art, but also AI-based digital personhood.

While it is rarer to find an AI robot-artist that specializes in music rather than in visual arts, it is not impossible. Alter3, the result of a collaboration between researchers at Osaka University and the University of Tokyo, is an autonomous humanoid robot that can sing and conduct human orchestras. In 2020, Alter3 conducted a live orchestral performance in Sharjah, United Arab Emirates of an “android opera” entitled Scary Beauty written by composer Keiichiro Shibuya.

As a conductor, Alter3 expertly mimics the techniques of a conductor, except it’s physique, a gangly metal body and hyperrealistic face and hands, creates an uncanny valley feeling.  Regardless, the robot bounces and spins around its base in such a way that is reminiscent of an enthusiastic human conductor. As an autonomous AI conducting robot, Alter3 analyzes human musical compositions and uses artificial intelligence to determine the tempo and dynamics in which it will direct the human orchestra to play. While conducting is certainly a method of creating art, Alter3 is less successful at being perceived as an independent artist because the general public is unaware of the complexities of a conductor’s job and its reliance on thorough, sometimes subjective, musical interpretation.

Case Study: Shimon

Shimon is an artificially intelligent singer-songwriter robot (formerly an artificially intelligent marimba-playing robot) created at the Georgia Institute of Technology. “He” uses deep learning to compose his own melodies based on generative and rules-based algorithms trained on jazz, progressive rock, and music theory.  He writes his own lyrics based on two starter words given by creator Gil Weinberg, which is supported by the semantic understanding of English words.  He performs these compositions by playing the marimba and singing along with a voice generated through deep learning. The project engineers updated Shimon’s face last year to have a mouth and eyebrows that move as a human’s does, so it would more accurately mimic the humans with which he collaborates during improv jam sessions.

Although Shimon receives human assistance from Weinberg, who arranges Shimon’s melodies into full-length songs, his artificial creativity might even make Ada Lovelace raise an eyebrow. He writes lyrics are oddly poetic and emotionally nuanced, and his voice is quite impressive, considering that it is generated using algorithms instead of using a pre-recorded human voice. In 2020, Shimon released an album of his compositions, which is available to stream on Spotify, and has posted several music videos on YouTube. Despite his non-humanoid appearance, Shimon is easy to anthropomorphize due to these human-like achievements.

Conclusion

Robots will not replace humans in the arts anytime soon, but they are increasingly working alongside us in a wide range of roles, which are made possible by their different attributes and capabilities – from intermediaries and tools to potential artists. In a post-pandemic world whose inhabitants have become accustomed to virtual experiences, these robotic arts roles, including facilitating virtual museum visits, performing through a computer program rather than a script, and creating what maybe approaching original and creative art, are probably here to stay.