Artificial Intelligence opens new avenues for museums to engage audiences, and create a plural vision of the museum. In our latest podcast Daniel Morena, of 32Bits, discusses the Iris+ AI exhibit integration used at the Museum of Tomorrow in Rio de Janeiro, Brazil.
This interview was recorded at a recent AAM convening Museums and New Intelligences.
Hello AMT Lab listeners, this is Jenee Iyer. On the newest installment of the tech in the arts podcast I am talking with Daniel Morena of 32Bits about Iris+, which is an AI powered experience integrated into the main exhibit at the Museum of Tomorrow in Rio de Janeiro. Enjoy!
Jenee Iyer (AMT Lab): Welcome to the arts management and technology podcast. This is Jenee Iyer and I am here with Daniel Morena of 32Bits, which is an interactive design studio out of Brazil, and we’re here to talk a little bit today about some of their programs, particularly an in interactive AI program they designed for a museum called Iris. Daniel if you could tell us a little bit about Iris.
Daniel Moreno (DM): Iris is our first attempt to do [a] digital assistant for a museum. The idea of the software is to help users to navigate thought content. The museum is a technological museum, The Museum of Tomorrow, and they present a lot of digital information, and more than 2000 pages of content, so the idea is to create a digital assistant to aid people and help them make sense of continuous visitation on the museum so [that] the next time you go there and interact with the system it remembers what you already saw and suggest new things for you to see based on your previous choices. But none of this was yet artificial intelligence. Artificial intelligence came later in the museum […], as a side project which is a new installation at the end of the main exhibition of the museum. It is an installation we call Iris+, a place where Iris [is given] a voice interface and cognitive intelligence to talk back with visitors.
AMT Lab: So when a visitor comes to the museum, what is the experience like for them. What do they receive? What does the flow play out for them? What kinds of questions are they going to be answering or interacting with the Iris system?
DM: They will receive and RFID card, and when they use them in the main exhibition it summons up Iris. Iris presents itself and asks him to provide identification data so if the visitor opt in to play with her. It starts to follow up [on] whatever content the user is seeing and offers suggestions in a second and third visit. At the end of the exhibition the same card can be used in Iris+ to summon up this improved voice interface and webcam with which the visitor can really chat with her (the Iris+ AI) because she can offer provocations. She (Iris+) asks the visitor “after all you see in the main exhibition, what is your major concern in the world today about sustainability?” So the visitors are invited to talk about any subject, which is an innovation because AI tends to be a question answer system.
AMT Lab: So, I am really curious about this idea of having an AI ask open-ended questions. So, how did you get to the point of having AI ask an open-ended question to the user and the social reasons for that. From a data and technology perspective [what were] the challenges of having the AI ask an open-ended question to the viewer, so kinda [what were] the social implications and data challenges of that (developing AI capable of asking open ended questions)?
DM: I think the most interesting social implication is to take the visitor out of their comfort zone and suggest to him something to actually do to change things that are concerning him in the current situation in the world. So it is really engaging for them to experience a new technology which is not common in the sense the way Iris conducts the dialogue. She asks you something, as we are discussing, so there is a point where technically it was a challenge because all the systems draw for you to create this kind of experience – as question answer and tasks. So when you decide to do [. . . .] something [that is] enabling the visitor to respond to anything, we need to - the content team and the IBM team which was consulting with us, the system uses IBM Watson for driving conversation - there are a lot of challenges [for ] the intelligence because we needed to previously train for common subjects, and specialize it, and keep tracking. The museum staff keeps updating it her with more subjects and more ideas and the conversations are drawn in a way if someone tries to talk about pollution for example, which is a broad subject, the way dialogue is constructed makes Iris+ talk about a specific thing on the subject randomly [chosen] for a subject of three or four possible specific questions, and after that she tries to get from users something more specific because the idea in the final part of the installation is for a suggestion to an initiative for the visitor to actually do something so the more specific, the more easy it is to find an intent, [and] a corresponding initiative for that intent [. . . .] to change things, so if you draw from this pollution question about garbage [then] collecting garbage in order to increase recycling that will be a clear initiative in which the visitor could interact to change the situation for example.
AMT Lab: That is a real interesting way of kind of tying the museum and the purpose of the exhibit, and technology, and creating a way for audience to then go have a real world practical application with practical tips based on the questions they have been asking and interacting with the AI. It is a real interesting engagement aspect. Are there any other engagement aspects you have built into the experience? I know you were showing some pictures earlier during the conference about a wall where you could examine all the different members of your community who are also interested and engaged in these social issues, and the kind of aggregation that happen from that.
DM: The main idea of this, because the experience is divided in two moments, one moment you will talk to Iris+, and a second moment where you’re going to see how your concerns relate to the previous visitors concerns [while] interacting with that installations. So when you go there and are talking about garbage, or pollution, or anything which comes to your mind as a concern [. . . . ] at the end of this conversation, and you have your recommendations, you are invited to join this data visualization. So that was a design challenge because the museum has a really broad public. We have a small child visiting the museum for the first time as well as a teacher who teaches physics, or a post doc writing thesis about physics, real scientists, and kids in the same public demographics. We tried to come up with visualization idea which could attend both the demographic extremes equally, so we choose to make a more playful visualization, and create this sense that we are a community. The design created a constellation of particles, and this constellation of particles are connected randomly by proximity so one particle is close enough to one another to form a connection, and the color of particles illustrate concerns, a group of colors are associated with direct sustainability things, and another group of colors are concerned about questions regarding social concerns, planet, social, and human evolution questions. When you see this you see a beautiful continuous movement of constellation of things. You don’t know exactly what it is it, but if you keep looking at it the particles [which] are alternating for focus, and when I focus you can see that was a person, because the name of the person, and the concern she had when they interacted with this. By the color system at the glimpse of the wall, you have an idea if people are more concerned about questions of sustainability or society, social concerns, or technological or biological evolution. Afterwards you take the dialogue and the self-interferes in the visualizations. So people see her or himself on the wall and the particles are organized in simple graphics of a sphere, so part of the particles are dimmed, to illustrate people with different concerns from yours and then a full [] particle to represent another visitor with your same concern. At a glimpse you have a sense, a feeling how much visitors have the same concern as you or different concerns. And your name is in a focused position. The particle is bigger, and your name appears. So you can see yourself in it. You can do something. It is not to be only to be a passive experience. Interactive ok, passive in the sense that you do nothing. You do nothing you just get impacted. You are invited to actually do something [with Iris+]
AMT Lab: It sounds like a really fascinating tool for engaging audience, particularly around exhibits that do have a social concern, or that have strong reactions from audiences. It sounds like a fantastic way to visualize information about audiences and sentiment. For arts managers and museum administrators what kind of data can this kind this of an AI integration with exhibits provide for them, and how can the museum utilize that data?
DM: In the case of Iris+ [it] has in their registration because Iris already registers people who visit her. We got their e-mail, name or his name, and first and last name. [This is sufficient] for tracking and offering a better, tailor made visitor experience integrating the experience with emails, so after your experience you will receive an e-mail asking about your experience and offering you things to explore or should explore in the next visit. Iris + takes it one step further and tires to identify where this visitor is located in the world, or Rio de Janeiro specifically, and tr[ies] to offer nearby initiatives (related to the exhibit) to where they live, and this will increase the engagement because it is not something where you have to go to another state to participate. It is nearby you so you have all the tools needed to go on and do something. This kind of demographic you can receive. We ask for the place people live generally, the neighborhood, state, country, and her or his age. In the sense we need to know if the visitor is over legal age to participate with his or her image, and that data tries to segment the concerns by age or location. So you have a good map of what kind of concerns people indifferent areas of the works, and majority in the state or area of Rio de Janeiro.
AMT Lab: Great. It sounds like a lot of very useful information for managers. When you look ahead, what is the future to you? What is that golden unicorn off in the distance that you are chasing?
DM: Right now it is trying to make a more customized way. The ultimate customization I think. For me the future of this AI technology applied to museums, as I think museums as a service for the community, is to help to build and to develop this community. In the sense this technology - with enough information - using this technology of artificial intelligence to find matches and to find affinities between visitors, and to try to build the community around the museum, try to make the vision of the museum a plural vision. You know. Not just people to visit and consume the content the museum offers, but to engage and create a community around it, because I think museums are not neutral nor should they be. The museums should stand for a point of view and I think they also have a community around it, and the beauty of this community should be facilitate[d] by this technology. And I think today you need a lot of effort to make this work, and I think in the future this will be more natural. So when you visit a museum and they use this technology it will be much more easy for you to connect with this mission, or point of view that museum its collection, and this actuation providing to you.
AMT Lab: Thank you very much for speaking with me today. I really appreciate your thoughts and input on the project that you have worked on and where you see museums and technology and AI technology’s in museums heading in the future.
DM: Thank you very much for the opportunity.
Note: The portions of this transcription marked in brackets and parenthesis have been edited for ease of reading. Small filler words may have been omitted in transcription.