Internet of Tattoos? NIST Workshop Plumbs Body Art Algorithms

In-brief: One in five adults in the U.S. sport body art, making tattoos a useful tool for identifying both criminals and their victims. A NIST workshop explored ways to use AI to better identify and catalog tatts. 

It used to be that the only place you’d see tattoos was at your local VA hospital. No more. In the last 30 years, body art has gone mainstream. One in five adults in the U.S. has one. There are reality shows centered on tattoo parlors and even full-sleeve stick-on tattoos so that even kids and the faint of heart sport that David Beckham look.

For law enforcement and forensics experts, this is a good thing; tattoos are a great way to identify both perpetrators and their victims. Given the number and variety of tattoos, though, how to describe and catalog them? Clearly this is an area where technology can help, but it’s also one of those “fuzzy” problems that challenges the limits of artificial intelligence.

That was the purpose of a recent workshop by the National Institute of Standards and Technology (NIST), which summoned experts from across the country and around the world to discuss the challenges and possible approaches to automated “tatt” recognition.

NIST’s Tattoo Recognition Technology Challenge Workshop challenged industry and academia to work towards developing an automated image-based tattoo matching technology.

Currently, cataloging tattoo images for sharing is a manual process that relies on keywords- obviously a system with lots of room for error or differences in interpretation.

Participating organizations in the challenge used a FBI -supplied dataset of thousands of images of tattoos from government databases. They were challenged to develop methods for identifying a tattoo in an image, identifying visually similar or related tattoos from different subjects; identifying the same tattoo image from the same subject over time; identifying a small region of interest that is contained in a larger image; and identifying a tattoo from a visually similar image like a sketch or scanned print.

According to NIST computer scientist Mei Ngan, “state-of-the-art algorithms fared quite well in detecting tattoos, finding different instances of the same tattoo from the same subject over time, and finding a small part of a tattoo within a larger tattoo.”

However, they struggled to detect visually similar tattoos on different people and matching a tattoo image from a sketch or sources other than a photo.

“Improving the quality of tattoo images during collection is another area that may also improve recognition accuracy,” Ngan said.

Among the systems demonstrated was “GARI” – A Gang Tattoo and Graffiti System (PDF) developed by Purdue University that collect gang graffiti and gang tattoos images, analyzes and interprets them, and provides access to law enforcement via a mobile application.

A submission by MITRE detailed that company’s efforts to use Scale-invariant feature transform (SIFT) algorithms to analyze tattoos by breaking images down to a set of key-points (for example corners and edges) to create a descriptor that can be compared to key points from test gallery images, and determine similarities (where the more point matches indicates better matching images). The method worked well in some situations – such as close matches or highly biased comparisons, but not as well with others – such as similarly shaped but clearly distinct images.

Six organizations participated in the challenge: Compass Technical Consulting, LLC.; the Fraunhofer Institute of Optronics, System Technologies and Image Exploitation; the French Alternative Energies and Atomic Energy Commission; MITRE; MorphoTrak and Purdue University.

Of course, the advent of powerful tattoo recognition tools could pose serious privacy concerns, as well. Coupled with inexpensive technologies like IP enabled surveillance cameras and drones, tatt recognition tools could make it easier for oppressive regimes to identify individuals and monitor their movements. Groups like the Electronic Frontier Foundation have warned of the movement towards “a pervasive surveillance state” in Latin American countries and elsewhere, where drones and closed circuit cameras pose “significant concerns for privacy and civil liberties.”