Facial surveillance technology falsely identifies 28 members of US Congress

By agency reporter
July 28, 2018

The American Civil Liberties Union Foundation of Northern California has released the results of a test revealing that Amazon’s facial surveillance product Rekognition falsely matched 28 current members of Congress with images in an arrest photo database. 

In the ACLU’s test, legislators from both parties, from states across the country, were falsely matched. Congressional members of colour were disproportionately identified incorrectly, including six members of the Congressional Black Caucus. Several months ago, the Congressional Black Caucus sent a letter to Jeff Bezos that expressed concern about the “profound negative unintended consequences” face surveillance technology could have for African Americans, undocumented immigrants, and protestors.

The ACLU used Rekognition’s default settings to compare every member of Congress with 25,000 public arrest photos, and verified the results with an independent expert. The test mirrors how a sheriff’s department in Oregon is deploying Rekognition to compare people’s faces against a mugshot database.

“Our test reinforces that face surveillance is not safe for government use,” said Jacob Snow, Technology and Civil Liberties Attorney at the ACLU Foundation of Northern California. “Face surveillance will be used to power discriminatory surveillance and policing that targets communities of colour, immigrants, and activists. Once unleashed, that damage can’t be undone.”

These results follow a nationwide movement in protest at the government use of face surveillance, prompted by the ACLU’s release of a report showing how Amazon is marketing facial surveillance to law enforcement. 

Since the release of the documents, the ACLU has delivered over 150,000 petition signatures to Amazon’s headquarters in Seattle demanding that Amazon stop selling facial surveillance technology to governments. A coalition of nearly 70 civil rights organisations, hundreds of academics, as well as Amazon shareholders and employees, have all expressed grave concerns about the profound civil liberties and civil rights threats posed by facial surveillance.

“Congress should press for a federal moratorium on the use of face surveillance until its harms, particularly to vulnerable communities, are fully considered,” said Neema Singh Guliani, ACLU legislative counsel. “The public deserves a full debate about how and if face surveillance should be used."

Amazon responded to the ACLU’s test by arguing that the ACLU should have used higher threshold settings than Amazon’s own default. 

Jacob Snow, technology and civil liberties attorney at the ACLU Foundation of Northern California, responded with the following comments:

“Amazon seems to have missed, or refuses to acknowledge, the broader point: Face surveillance technology in the hands of government is primed for abuse and raises grave civil rights concerns. It could allow – and in some cases has already enabled – police to track protesters, ICE to continuously monitor immigrants, and cities to surveil their own residents. Changing the matching threshold doesn’t change the danger. In fact, it could exacerbate it.

“In addition to remaining silent on these very real concerns that members of Congress, community groups, and Amazon’s own employees, shareholders, and consumers have raised repeatedly, Amazon is acknowledging that Rekognition – a product that it aggressively markets to law enforcement – can and will misidentify people by default. That’s downright dangerous, and there’s more: We know from our test that Amazon makes no effort to ask users what they are using Rekognition for. Instead, Rekognition sets one default: the same 80 per cent we used in running our test.   

"We also know that Amazon’s website, right now, shows the use of an 80 per cent confidence for recognizing human faces. It shows that Amazon is recommending an 80 per cent confidence score in ‘Face-Based User Verification.’ If an 80 per cent threshold is not 'appropriate for identifying individuals with a reasonable level of certainty', why is Amazon highlighting that confidence level for recognising human faces?

“Amazon has effectively admitted that its product is dangerous out of the box. This just reinforces that Amazon needs to take greater responsibility for Rekognition. And Congress needs to take action to put on the brakes and enact a moratorium on law enforcement use of face surveillance.”

Following the ACLUS's experiment, some of the members of Congress whose faces were falsely matched have written to Amazon CEO Jeff Bezos to demand a meeting to discuss Rekognition. 

* American CIvil Liberties Union https://www.aclu.org/

[Ekk/6]

Although the views expressed in this article do not necessarily represent the views of Ekklesia, the article may reflect Ekklesia's values. If you use Ekklesia's news briefings please consider making a donation to sponsor Ekklesia's work here.