The advancement in artificial intelligence and machine learning has contributed to the growth of computer vision and image recognition concepts. Image recognition, as one of the subsets of computer vision, is the process of identifying objects, places, or people in images or videos. The concept has been applied widely in areas including facial recognition, license plate matching, pattern matching, and so forth. In order to increase audience engagement, many museums are also teaming up with technology companies in the field to create innovative applications.

Case Study

The Barnes Foundation in Philadelphia, for instance, utilized an image recognition tool to enhance the visitor experience. The tool, Barnes Focus guide, works on a website (barnesfoc.us).

On the Barnes Focus guide website, the visitor can use the camera to scan an artwork such as an oil painting, a statue, or an artifact. The program only requires a small segment of the image to recognize it and direct the user to the result page. The user can see an image of the painting with a text narration, including the title, artist, date, medium and dimensions.

The Barnes Focus guide application provides an enhanced visitor experience in that the Barnes Foundation never labels the paintings because of the founder, Dr.Albert C. Barnes’s insistence. The wall labels are removed and most of the pictures are placed symmetrically, remaining precisely in the position Barnes had placed them, letting the color and compositions play against each other. Though it’s aesthetically pleasing, for an audience who is not familiar with impressionist art, the layout can be confusing. It’s hard to distinguish artists from one another and recognize each artwork. With Barnes Focus guide, visitors can quickly scan images and find additional information, which is much more convenient than manually looking up pamphlets on the shelf.

Feasibility

Although the integration of image recognition and exhibition can generate synergy as in the Barnes Foundation’s case, this kind of approach may not apply to other arts organizations. This may be especially true for institutions with extensive labels, audio guides, docents, guided tours, and so forth. The easier the information can be acquired by visitors currently, the smaller the marginal return for the museum to invest and incorporate image recognition into their permanent exhibition as the information provided by the new tool would likely be redundant. Museums, however, can regard image recognition as an add-on feature and create a more interactive experience with it.

Applications

The National Portrait Gallery in Australia, for example, teamed up with catchoom, a visual first technology vendor that mainly serves retailers, to create their interactive application ‘Headhunt!’. Instead of informing all the visitors in the museum, the app is specifically designed to attract youngsters, allowing children “to take pictures of the portraits in the gallery with an iPad and instantly access interactive learning experiences, turning the visit into a fun game (Catchoom).” In this case, the aim of incorporating image recognition is not only to provide general information regarding exhibits but also to help intuitively engage the younger generation with modern technology.

Another solution for museums is to encourage visitors to utilize a third-party application rather than developing their own. For instance, Smartify, an app launched recently, uses image recognition to identify scanned artworks and provide people with additional information about them. It has partnered with some famous museums such as the Louvre and the Met, gaining access to the organization’s digitalized collections’ data. A benefit of using a third party developed app is that it can work across institutions. Users can add the works from different institutions to their own digital collections. Museums simultaneously save resources since there is no need to build one themselves. Otherwise, external support from donors and foundations are required to make a program possible. Barnes Focus, for instance, is funded by the Knight Center for Digital Innovation in Audience Engagement at the Barnes.

Pros and Cons

There are various pros and cons regarding the usage of image recognition tools in the museum. Firstly, the information provided by the tool is typically precise and quickly generated, as one of the strengths of artificial intelligence is its ability to process huge amount of data and retrieve it in an efficient way. Secondly, visitors can have access to more information regarding artworks they are interested in. For instance, in a typical exhibition, longer texts are shown on labels of more “significant” artworks, while only titles, author, and date are given for the others. Digitalization generates an equal chance for every work to be introduced in detail since there is no space or word limit. Thirdly, it can help the organization engage and educate the audience by inspiring them to explore the exhibition with the image recognition tool.   

Nevertheless, image recognition tools are all based upon digital devices, and the usage of electronics, especially mobiles, in museums is a controversial issue. Visitors might focus too much on mobile phones and end up ignoring the details shown on the objects such as the texture, real color, and brushstrokes of a painting. Additionally, even though many of the tools are user-friendly for tech natives, it’s still questionable whether it enhances the experience for the “digital immigrant”. When adopting and applying the tool, the institution should be careful to prevent people who are not familiar with technology from feeling excluded – just like what the Barnes Foundation does by providing pamphlets for those who don’t have a mobile device.  Finally, the museum should consider if the application of image recognition can generate a measurable return that outweigh the setting-up cost. If so, a third-party application might be more appropriate than developing an original version.

In conclusion, as an increasing number of museums and galleries digitalize their work, it’s trendy to have all the information online. Image recognition is helpful for visitors to access information in a precise and quick manner, but the question is how to tailor and incorporate it properly into an exhibition and daily operations of an institution so that it won’t impair the audience experience or the sustainability of the museum. It will be interesting to see what the future of image recognition tools will be like in museum settings.

Resources

“An Interactive Museum Experience for National Portrait Gallery.” Catchoom. Accessed December 18, 2019. https://catchoom.com/case-studies/headhunt-app-turns-a-museum-visit-into-an-interactive-experience-with-image-recognition/.

Gupta, Shubham. “Understanding Image Recognition and Its Uses.” eInfochips, May 16, 2019. https://www.einfochips.com/blog/understanding-image-recognition-and-its-uses/.

Reynolds, Matt. “Image Recognition App Scans Paintings to Act like Shazam for Art.” New Scientist, March 3, 2017. https://www.newscientist.com/article/2123373-image-recognition-app-scans-paintings-to-act-like-shazam-for-art/.

“The Barnes Foundation.” Barnes Foundation. Accessed December 18, 2019. https://www.barnesfoundation.org/press/in-the-news/the-new-barnes-foundation-building.