On November 14, 2019, a group of protesters were dressed in white jumpsuits, covering themselves from head to toe in Washington, DC. Smartphones on their foreheads were randomly scanning people passing by, without any consent. They live-streamed the process, aiming to attract the attention of lawmakers and appeal a ban on facial recognition technology. This technology is in a predicament, and has been there for quite a while. How did it become the center of such controversy and how does such controversy influence the art industry?
What is facial recognition technology?
Beginning around the 1960s, facial recognition technology first emerged in the United States. Woodrow Wilson Bledsoe, developed a manual system to match pictures of faces by coordinating different facial features in grids. This was the very early stage of the facial recognition development and marked a significant step to the eventual development of biometric technology. Since the huge improvement in computer science toward to end of the century, this technology has been improving faster and smoother, and achieved its milestone in 2001 with its unprecedented public surveillance system at the Super Bowl. The world’s attention was captured.
Modern facial recognition technology integrates biometrics modalities to capture and analyze facial data. 3 steps are taken in this process:
1. Capture face from photos or videos. The computer learns where the face is by working with a trained algorithm, like deep neural networks, to estimate the position of the face.
2. Analyze facial landmarks and geometric features. Facial landmarks are points and patterns on face, which create unique structures and develop a ratio-based face model. As 68 facial landmarks are measured, prominent geometric features reduce face search spaces and generate facial data, like the distance between the eyebrows and the width of the noses, to distinguish each individual’s face.
3. Compare to the database and match the potential candidate. The computer then compares the facial data to the database, using different matching techniques to see if there is a matching candidate.
The basic facial matching technique includes 1:1 and 1:N, designed for different situations. 1:1 is usually for authentication, by comparing the captured data to a single known identity to verify if the input matches the stored data. Unlocking digital devices with faces is an example of 1:1 matching. 1:N method, used as identification, mostly helps police to seek and locate a suspect by matching the suspect’s photo to the vast candidate data.
Figure 1: Facial Landmark Points. Source: Facial Landmark Detection by Deep Multi-task Learning.
IMPLEMENTATION OF Facial Recognition Technology
A wide range of implementations of facial recognition technology exists for different purposes. The technology is not only tied to face unlock for mobile phones and computers, which many possess and utilize today, but also is used to assist the FBI’s video surveillance streaming system with more sophistication. The widespread use of this technology makes “facial recognition” a buzzword as it becomes ubiquitous in people’s daily life. Categorizing its design and implementation, the facial recognition technology is placed in 3 major areas: security, retail and social media.
Security
Facial recognition technology can work for both individual security and public surveillance, and can be implemented almost everywhere we can imagine. Take the airport as an example, the U.S. Customs and Border Protection agency is expanding the use of facial recognition technology to replace the current password verification and human check-in method. Customs will match up passengers’ images with passports, tickets and other personal information to verify their identifications. The technology is speeding up the airport security process. Furthermore, a missing child can be recognized and located using this technology as long as the airport can access the necessary database. Public crimes could also be reduced by utilizing the technology.
Retail
Internal theft, shoplifting and violence happening in retail chains and grocery stores increase losses and damage the reputation of a company, causing certain retail shrinkage. Surveillance cameras works great for deterring those types of illegal behaviors: retailers using facial recognition technology reported a 34% decrease in shoplifting and 91% decrease in workplace violence.
Online payment today is working closely with our faces as well. Alipay, in China, connects facial recognition technology with individual online payment, allowing people the ability to purchase goods from vending machines without cash or card. The only thing people need to bring to the machine is their face-- to trigger the individual online credit system-- which seems to be easy and efficient.
Social Media
Tagging people in photos is much easier today, thanks to the help of facial recognition technology. It can automatically detect facial information on each photo and connect the potential people from the related friend database. Tech giants, like Facebook, have been continually training their Artificial Intelligence in facial recognition by encouraging users to participate in any face-related task and feature as they collect data from users in the process. Their technology might be more powerful and accurate than the government’s. Facebook’s algorithm, DeepFace, scores 97.25% accuracy, while the FBI’s accuracy rating is roughly 85%.
Figure 2: Facebook tagging system. Source: Digital Trends.
Facial recognition and public responses
The practical applications of facial recognition technology will create a different life, and to some extent, an easier and safer one than before. Skeptical voices, nevertheless, pop up and drag more and more people into the realistic pessimism concerning the potential risks of facial recognition technology. People started to question how organizations would work with it and what they would do with the collected data. The barriers between face owners and collectors were blurred, blowing transparency away. Owners are rarely told the length of time in which data would be kept and what will happen with it after its purpose has been fulfilled. Based off a survey conducted by Pew Research Center, only 36% Americans trust tech companies to use facial recognition technology and only 17% trust advertisers to do so. Face panic grows wilder and out of control.
Figure 3: Percentages of Americans’ trust in facial recognition technology. Source: WIRED.
It is surprising that there is no current federal law specifically regulating this technology. State law play a different role in regulating technology issues, and vary from state to state. 5 states are seeking to limit the use of facial recognition technology and 3 cities— San Francisco, Oakland and Somerville, Massachusetts— have just banned the government use of this technology. On Jan 7, 2020, the White House proposed guidelines to govern the use of artificial intelligence, especially facial recognition, since the previous principles may be too vague. The guidelines urge regulators to put transparency, fairness and risk assessment into consideration, encourage public feedback throughout the process until it is formally approved.
China is another country holding one of the world’s most powerful facial recognition systems. Widely expanding its facial recognition technology network since 2015, China is one of the countries with the largest number of surveillance cameras in the world. It is estimated that the number of surveillance camera will grow to 626 million in public and private spheres by 2020, keeping watch over nearly 1.4 billion people. The county’s sophisticated facial recognition system leads the society to an advanced monitoring level, fitting its political system and empowering China’s architecture of social security.
Because so much national praise for the technology’s power exists in China, Chinese people have been losing face panic since its initiation: the government promotes the successful use of facial recognition in national media, convincing people of its thrilling capabilities in enhancing criminal identification, public safety, social responsibility, etc. The avoidance of revealing facial recognition’s negative social effects became the anesthesia leading the pluralistic ignorance about the technology. Nevertheless, the first court case about the enforcement of facial recognition in a wildlife park and the data security risks regarding the ZAO face-swapping app has brought the ethical problems of the technology to light in China and triggered nationwide debate recently. People are starting to discuss the data and privacy concerns. On January 1, 2020, China put into effect the Cryptography Law, which regulates cryptography for both government and private use. The law classifies cryptography into 3 categories-- core cryptography, ordinary cryptography and commercial cryptography-- and put them all under the supervision and random inspections of the Ministry of State Security and the Communist Party. Although facial recognition technology does not directly meet the definition of cryptography in this law since it does not “apply specific transformation”, the implementation and management of the biological identification information still requires encryption technology in the process. It will be interesting to see how this new law regulates the aggressive use and development of facial recognition in China in the near future.
Contrary to absolute supervision, the European Commission strives to maximize citizens’ rights over facial recognition data and limit the discriminate use of the technology. The General Data Protection Regulation (GDPR) which came into force on May 28, 2018, aims to give owners control over their personal biometric data, providing them explicit information about the purpose of the data collection and the length of time the data will be held. All facial recognition activities cannot be processed until achieving consent from the owners. However, the increasing roll out of facial recognition technology use among the police poses a dramatic threat and fear to the public. In addition, the 81 percent error rate of the UK’s police facial recognition system landed innocent people in unnecessary dilemmas and diminished people’s trust. To reshape the current implementation of the facial recognition system, the European Commission plans to rebuild regulations for this technology to protect citizens from indiscriminate supervision—On January 17, 2020, according to a draft white paper, the European Commission is considering a ban on facial recognition used in public areas for up to five years. If this time-limit ban is implemented, the researchers and regulators would be given time to study the technology and establish the new legal framework to better regulate facial recognition in the region.
Although technology is changing fast, policymakers are trying to work hard to keep pace with technology developments. The novelty of the technology as well as the conflict of voices provide regulators with various potentials to explore the best way to implement this technology in real life.
Facial recognition technology and museum surveillance
Compared to other industries which have been using this controversial technology for decades, the arts and cultural industry seems to be a newcomer to this face-governed world. Although some museums have just tested it out, they are still meeting the same dilemmas other industries face. For example, World Museum Liverpool used facial recognition technology on every visitor as surveillance at a loan exhibit that occurred in 2018. The technology served as an extra security method to scan and catch any visitor who damaged or behaved hostile toward the displayed artifacts. Clear warning signage was placed around the exhibition space to communicate the legal use of the cameras. Although no visitor stood out and debated the use of surveillance cameras in the space during the exhibition period, a private campaign group, Big Brother Watch, revealed and condemned the dark side of the use of facial recognition technology a year later.
Figure 4: Locations of private sites that have used or are currently using facial recognition cameras in UK. Source: Dailymail.
People act differently toward museum management methods since there are varying levels of awareness, attitudes and knowledge of this type of technology. A similar situation happened at the Palace Museum in China. The museum was recently equipped with 3,300 plus CCTV cameras, covering the 7,750,000 square feet museum space. Cameras are installed to prevent visitors’ adverse behavior towards the cultural heritage as well as the intentional and negligent fire risks. Controversial voices appeared from some critics, arguing that such deployments in the museum environment are unethical and ineffective.
These public controversies in the museum space have raised public awareness of the emergence of facial recognition technology in the arts and cultural sector. It is reasonable to believe that there could be other arts organizations using similar technology secretly, without publicly disclosing its use.
What can facial recognition actually bring to museum operations?
There is no definitive answer as to whether or not surveillance systems or other facial recognition technology should be used in the museum space. Some people hold negative attitudes towards it since the aggressive technology is rather intrusive and offensive against basic civil rights, raising personal privacy issues. However, arts managers value the surveillance system, for security benefits are provided and suspicious activities are monitored. Facial recognition is the reflection of the museum’s respect to different entities: We care about you, so we are watching you. Museums value every artist and their art work in the space, striving to protect them from any damage; at the same time museums value visitor’s rights and should protect them from unnecessary implications from any unfortunate accident that may occur. Standard warning information might also serve to deter potential crimes.
There are many reasons why facial recognition technology is becoming increasingly significant to arts managers beyond surveillance. Museums can use facial recognition technology for the following purposes:
Artwork protection. This might be the main reason why museums want to have this contentious technology. Scanning visitors’ faces can prevent heightened security risks and accidental losses during exhibitions. Visitor safety would be enhanced, and crime rates would be reduced.
Visitor demographic analysis. Currently, most museums are conducting visitor demographic analysis with zip codes and surveys. With facial recognition in the space, the museum might be able to gain a clearer image to answer the WHAT question about their visitors. Age, gender, ethnicity or even height, weight and possible relationship can be known through analyzing the data from the facial recognition technology.
Visitor engagement analysis. Effective facial detection helps museums track visiting experiences in the space. Facial recognition technology might analysis visitors’ interests and attitudes by detecting their facial expressions and length of time spent at each work. The system might also conduct eye-tracking to figure out where visitors are most likely to place their attention. From the visitor analysis, museums can discover which art piece generates the most attention or close inspections and which one might need to be replaced. An individual’s visiting route is likely to be traced. This data would affect curatorial decision making and provide a more solid background for creating any event or program relating to the work and space.
Visitor art experience report. Museums could utilize facial recognition system along with other AI tools to generate individual art experience reports for visitors based on the data collection and analysis, similar to Spotify’s annual user report “Spotify Wrapped.” Visitors could easily get to know where they linger longer, which section might be their favorites and which art piece they miss in the journey. It might be easier for visitors to plan their visiting route for next time. More data information could be available to members since museums could get accumulated data from them and could be used to track reoccurring visits.
Admission. It might not be easy for every visitor to purchase a ticket with their face information, but for museum members and regular visitors, facial recognition technology might speed up their ticket purchasing time.
Employee management. Instead of using traditional clock-in and clock-out systems with fingerprints, there is the potential for museums to encourage employees to record their shifts with faces.
Conclusion
These possibilities create smart operational alternatives and introduce emerging technology into the museum environment. Note that there are still some precautions that need to be taken into considerations before informing visitors and employees to get their permissions. The objectives of the data caption might be the top information they would care about; the length of time in which an organization would keep the data and whether the collectors would sell or share facial data to others are all essential details to relay to the public. Seeking advice from the police and local council advisors to get further confirmation is essential to all arts institutions.
Controversial discussion never stops a new technology, and instead, it creates a buffer to help the aggressive technology to slow a bit. With this opportunity, arts managers and other stakeholders may be able to check if they are on the right track dealing with the technology and consider whether they need to reshape it to meet future challenges.
Resources
“A Guide to GDPR Data Privacy Requirements.” GDPR, 22 Feb. 2019, https://gdpr.eu/data-privacy. .
Brandom, Russell. “How Should We Regulate Facial Recognition? We Asked the Experts.” The Verge, August 29, 2018. https://www.theverge.com/2018/8/29/17792976/facial-recognition-regulation-rules.
Charr, Manuel. “What Implications Are There With Facial Recognition Technology For Museums?” MuseumNext, September 25, 2019. https://www.museumnext.com/article/what-implications-are-there-with-facial-recognition-technology-for-museums.
Chen, Angela. “The EU Might Ban Facial Recognition in Public for Five Years.” MIT Technology Review, January 21, 2020. https://www.technologyreview.com/f/615068/facial-recognition-european-union-temporary-ban-privacy-ethics-regulation.
“Facial Recognition Technology Code of Conduct Call.” BBC, October 31, 2019. https://www.bbc.com/news/uk-wales-50251643?intlink_from_url=https://www.bbc.com/news/technology&link_location=live-reporting-story.
Lange, Christy. “Surveillance, Bias and Control in the Age of Facial Recognition Software.” Frieze. Accessed December 11, 2019. https://frieze.com/article/surveillance-bias-and-control-age-facial-recognition-software?language=de.
Martin, Nicole. “The Major Concerns Around Facial Recognition Technology.” Forbes, September 25, 2019. https://www.forbes.com/sites/nicolemartin1/2019/09/25/the-major-concerns-around-facial-recognition-technology/#6760049a4fe3.
Metz, Rachel. “San Francisco Just Banned Facial-Recognition Technology.” CNN, May 14, 2019. https://www.cnn.com/2019/05/14/tech/san-francisco-facial-recognition-ban/index.html.
Mozur, Paul. “One Month, 500,000 Face Scans: How China Is Using A.I. to Profile a Minority.” The New York Times, April 14, 2019. https://www.nytimes.com/2019/04/14/technology/china-surveillance-artificial-intelligence-racial-profiling.html.
Mozur, Paul, and Aaron Krolik. “A Surveillance Net Blankets China's Cities, Giving Police Vast Powers.” The New York Times, 18 Dec. 2019, https://www.nytimes.com/2019/12/17/technology/china-surveillance.html?smtyp=cur&smid=tw-nytimes.
Roberts, Jeff John. “The Business of Your Face.” Fortune, August 14, 2019. https://fortune.com/longform/facial-recognition.
Rossenblatt, Joel. “Facebook Face-Scan Fight Nears Trial With Billions at Stake.” Bloomberg. Accessed December 11, 2019. https://www.bloomberg.com/news/articles/2019-08-08/facebook-must-face-class-suit-over-face-recognition-court-says.
Samuel, Sigal. “Activists Want Congress to Ban Facial Recognition. So They Scanned Lawmakers' Faces.” Vox, November 15, 2019. https://www.vox.com/future-perfect/2019/11/15/20965325/facial-recognition-ban-congress-activism.
Satariano, Adam. “Police Use of Facial Recognition Is Accepted by British Court.” The New York Times, 4 Sept. 2019, https://www.nytimes.com/2019/09/04/business/facial-recognition-uk-court.html.
Taylor, Josh. “Plan for Massive Facial Recognition Database Sparks Privacy Concerns.” Guardian News and Media, September 28, 2019. https://www.theguardian.com/technology/2019/sep/29/plan-for-massive-facial-recognition-database-sparks-privacy-concerns.