CrowdOptic - London Olympics

Meet The Software That Helped Catch The Boston Bombers

With one suspect in the Boston Marathon bombings dead and another on the run IN CUSTODY!  the global, collective effort to identify those responsible for the crime has ended, and focus shifted to apprehending PROSECUTING Dzhokhor A. Tsarnaev, 19. He and his older brother, Tamerlan Tsarnaev, 26,  were the subject of a massive manhunt, culminating in a firefight in the suburb of Watertown, Massachusetts, that killed the older Tsarnaev brother and set of a massive, daylong manhunt that shut down the metropolitan Boston area.(*)

CrowdOptic - London Olympics
A new generation of technology for mining data from smart phones and other devices could make crowdsourcing much more powerful.

So how did crowdsourcing fare in the effort to catch the two? You’d have to say: not too well. High-profile collaborative efforts to crowdsource public images of the Boston Marathon bombing site, like those organized by the group 4Chan, assembled intriguing collections of material and clocked impressive pageviews (3.4 million and counting). In the end, those efforts yielded some clues: the type of clothing worn by the suspects, as well as new and unseen photos of the two at the scene of the bombing. Mostly, though, they sowed chaos and confusion, accelerating the spread of inaccurate information and fingering innocent spectators as possible bombers. None of the “suspects” singled out by crowdsourced analysis as “suspicious” are believed to have played a role in the attack. Meanwhile, the folks who were incorrectly identified as by the ersatz analysis caused big problems for those who were singled out found their images on the front page of The New York Post labeled as suspects.

Even after photos of the two were released, misinformation flowed quickly from sites like Reddit to more mainstream news sources, including speculation that one of the bombers was missing Brown University student Sunil Tripathi. A whole section on the site Reddit emerged just to clear the names of those falsely named as bombing suspects.

But the ad-hoc methods used by the general public don’t mean that crowdsourcing doesn’t have a bright future. The fact may be that the happy medium is to leverage the reams of data from the public and then give authorities the tool to make sense of it. And that future may rely on a coming generation of sophisticated data analysis tools that leverage a flood of metadata from smart phones and other devices and use that to map the behavior of large crowds – monitoring passive information from spectators to pinpoint unusual “events” with amazing accuracy.

One example of that is technology by the firm CrowdOptic, which sells what it calls “focus-based services.” For the most part the applications are commercial: advertisers want to know if their ads, or billboards or commercials are getting the attention of viewers. CrowdOptic can collect and analyze a wealth of data from smartphones to help determine that – noting where smart phone cameras are pointed and correlating that with social media and other information.

CrowdOptic _ London Olympics
CrowdOptic can help investigators identify members of a crowd who might be focused on a point of interest.

“Our vision of the future is new apps that dynamically adapt based on knowing what activities people in a crowd are watching and engaging in, as well as joining people with shared interests together, right there in the moment,” said Asif Khan, Founder and President of the Location Based Marketing Association (LBMA), which is partnering with CrowdOptic.

But as has been noted (by me, actually), the technology also has huge implications for law enforcement or any organization that needs to monitor the behavior of thousands of individuals in a crowd: spotting disturbances (fights, explosions, etc.) almost in realtime.

Specifically, CrowdOptic can extract compass information from the EXIF information associated with images, and then use triangulation and other algorithms to identify “points of focus.”

“Send me 100k images of the Super Bowl and in 1 second (of) server time I can send you the picture/s containing (for example) the halftime show wardrobe malfunction representing the most views,” CEO Jon Fisher explained in an e-mail back in October.

Put in the context of the Boston Marathon bombing, that kind of tool can help investigators to mine metadata from photos. Instead of asking eagle-eyed Redditors to look for the “suspicious guys” or the “guys with backbacks,” tools like CrowdOptic can rapidly create a kind of mosaic of photos capturing a moment in time and their “focus.”

The idea is to go beyond mere “geolocation” (that is: who was in a specific area at a specific point in time). The goal of “focus awareness” is to map the shifting momentum in crowds by observing the individual focus of the members of that crowd.

Technology Clusters

CrowdOptic’s technology played a key role in helping authorities to sift through the photo evidence collected from the bombing scene. In that situation, the bombs’ locations acted as a magnet for all other photos containing bomb location in the photographs of the area before and after the explosions. CrowdOptic’s technology was used to piece together that visual information and give investigators a time lapse not just of the scene, but of people who could have captured an image of the points of interest – even from some distance. That’s information that wouldn’t show up just by collecting geospatial data of those around the bombing site at the time of the blast.

While CEO Jon Fisher isn’t saying much about the Boston investigation, he’s profiled here, and talks about being a successful entrepreneur here, things being as they are, its likely that CrowdOptic is getting sniffed over by some larger players interested in the kind of “focus based services” that CrowdOptic offers. Stay tuned.

(*) Story updated to reflect the arrest of Dzhokhor A. Tsarnaev and add information on the role CrowdOptic played in the investigation.