Huge Survey of Firmware Finds No Security Gains in 15 Years

A survey of more than 6,000 firmware images spanning more than a decade finds no improvement in firmware security and lax security standards for the software running connected devices by Linksys, Netgear and other major vendors.*


When networking hardware giant NETGEAR issues a security advisory for one of its many products, the company is careful to include a bit of boilerplate text at the end, acknowledging the work of independent security researchers who often bring serious software security issues to the company’s attention.

“NETGEAR constantly monitors for both known and unknown threats. Being pro-active rather than re-active to emerging security issues is fundamental for product support at NETGEAR.”

Home networking stalwart Asus makes similar claims on its web page.

“We take every care to ensure that ASUS products are secure in order to protect the privacy of our valued customers. We constantly strive to improve our safeguards for security and personal information in accordance with all applicable laws and regulations,” the company claims.

Those are comforting words. But an extensive study of thousands of device firmware images by NETGEAR, ASUS and 16 other vendors suggests that there is little truth behind them; the security of device firmware is terrible and has not improved in any measurable way over the last 15 years, even as attacks on connected devices like home routers have spiked.

“Nobody is trying.”

“Nobody is trying,” said Sarah Zatko, the Chief Scientist at the Cyber Independent Testing Lab (CITL), a non-profit organization that conducts independent tests of software security. “We found no consistency in a vendor or product line doing better or showing improvement. There was no evidence that anybody is making a concerted effort to address the safety hygiene of their products,” she said.

Zatko presented the findings of CITL’s extensive study in Las Vegas on Friday on the sidelines of the Black Hat and DEF CON conferences at an event hosted by The Hewlett Foundation. CITL was started by Sarah and her husband Peiter (aka “Mudge”) Zatko. It bills itself as a kind of “Consumer Reports” for cyber security, partnering with that organization as well as The Ford Foundation, The Digital Standard and online payments firm Stripe.

Sarah Zatko is the Chief Scientist at CITL

The CITL study surveyed firmware from 18 vendors including ASUS, D-link, Linksys, NETGEAR, Ubiquiti and others. In all, more than 6,000 firmware versions were analyzed, totaling close to 3 million binaries created from 2003 to 2018. It is the first longitudinal study of IoT software safety, according to Zatko.

CITL researchers studied publicly available firmware images and evaluated them for the presence of standard security features such as the use of non-executable stacks, Address Space Layout Randomization (ASLR) and stack guards, which prevent buffer overflow attacks.

Security Blind Spots in Firmware

The results were not encouraging. Time and again, firmware from commonly used manufacturers failed to implement basic security features even when researchers studied the most recent versions of the firmware.

For example: firmware for the ASUS RT-AC55U wifi router did not employ ASLR or stack guards to protect against buffer overflow attacks. Nor did it employ a non-executable stack to protect against “stack smashing,” another variety of overflow attack.

CITL found the same was true of firmware for Ubiquiti’s UAP AC PRO wireless access points, as well as DLink’s DWL-6600 access point. Router firmware by vendors like Linksys and NETGEAR performed only slightly better on CITL’s assessment.

There were some bright spots. Almost all recent router firmware by Linksys and NETGEAR tested by CITL boasted non-executable stacks. However, those same firmware binaries did not employ other common security features like ASLR or stack guards, or did so only rarely, CITL data suggest.

Zatko said the CITL testing is by no means comprehensive. Just the opposite. In order to test so many firmware versions, CITL had to focus on security features that could be verified “at scale.” That ruled out checks that require more hands-on assessments, such as looking for hardcoded passwords, data and privacy leaks or exploitable vulnerabilities.

Cyber Independent Testing Lab Firmware Images 2018
CITL data on major vendor firmware images from 2018 show no evidence of progress on security measures. (Data courtesy of CITL.)

“Stack guards and buffer overflow protection are the canaries in the coal mine,” she said: basic protections that all software should employ. The absence of even basic protections suggests that the tested firmware may contain more serious vulnerabilities and that firmware security is years behind the security of applications like Windows, OS X or Google Chrome and FireFox.

“These are the seatbelts and airbags of the software world. These numbers are unheard of in operating systems or (Web) browsers. Its just a sign that they’re not trying,” Zatko said. 

Even worse, CITL researchers found no clear progress in any protection category over time, said Zatko. Researchers documented 299 positive changes in firmware security scores over the 15 years covered by the study…but 370 negative changes over the same period. Looking across its entire data set, in fact, firmware security actually appeared to get worse over time, not better, CITL said.

Cyber Independent Testing Lab router firmware security over time.
CITL assessments of router firmware from 2003 through 2018 show no consistent improvement in security.

Looked at longitudinally, the results of the CITL testing show no clear trend. “It’s pretty random looking,” Zatko told Security Ledger. In fact, security scores for the same vendor diverge over time rather than converge as the number of product lines the company supports increase, Zatko observed.

In an email statement, Linksys Vice President of Communications Jen Warren said that the company tests binaries for known security vulnerabilities and that “the security features described add no value if you have physical access to the router –these features help prevent exploitation of an application vulnerability that is exploitable over the network.”

Other vendors including ASUS and Ubiquiti did not respond to emails from Security Ledger seeking comment on the CITL research.

A Failure with many Fathers

CITL doesn’t point the finger at any single cause for the lack of security in firmware. Zatko said the persistent failure, across so many vendors, is the product of systemic failures in how software is produced.

“It’s a bottom up problem,” Zatko told Security Ledger in an interview. “Educators in (computer science) curricula don’t talk about this. Organizations that make software don’t talk to developers about it. There are no organizational rules about ‘You have to use these build flags’ or ‘You have to do this post build testing.'”

In many cases, enabling security features is as simple as enabling a small set of options during the software “build,” when firmware source code is turned into executable “binaries” that are shipped with routers, cameras, smart television sets and other hardware.

“It’s like every stage in the chain had a blind spot about these basic safety features that have been around forever so they took them for granted and assumed they’re there,” she said.

The poor scores suggest that many companies making devices for the Internet of Things have not adapted their practices to account for the increased security risks that accompany connected devices.

“These are companies that were primarily in hardware until recently,” Zatko said. “Mostly they’re doing the wrong thing.”

Mounting Attacks on Embedded Devices

The stakes are high. Attacks that target routers, cameras and other non-personal computer devices are on the rise. In 2016, the Mirai botnet, which was made up largely of compromised digital video recorders and Internet connected cameras, showed the power that tens of thousands of small, compromised devices could have.

Zatko said attackers are turning to those devices not for a challenge, but because they are easy prey, with software protections that are often years behind those in Windows, Apples OS X or on platforms like Google Chrome.

Despite that, Zatko said much of the conversation about IoT security focuses on software applications, but misses the security of firmware. Hardware vendors need to do much more to harden their devices prior to release. That means assessing the security of software development tools and implementing basic hardening consistently.

Such steps are easy to implement, will pay big dividends and are measurable, she said. There is no evidence any of the vendors studied by CITL plans to implement them.


(*) Updated to add comment from Linksys – PFR August 14 2019

70 Comments

  1. Pingback: Daily NCSC-FI news followup 2019-08-14 – Kurittu.org

  2. Pingback: Huge Survey of Firmware Finds No Security Gains in 15 Years – CS | Computer & Security

  3. Pingback: Survey of Firmware Finds No Security Gains in 15 Years #Security #InternetOfThings #IoT #Firmware @securityledger « Adafruit Industries – Makers, hackers, artists, designers and engineers!

  4. Pingback: Amazing how things (don’t) change | Chatroom Stories

  5. Pingback: Huge Survey of Firmware Finds No Security Gains in 15 Years - ZRaven Consulting

  6. Will pay big dividend how? And how do you measure their success?

  7. Pingback: SANS Daily Network Security Podcast (Stormcast) for Friday, August 16th 2019 - SANS Internet Storm Center

  8. Pingback: Pod biurkiem masz ciekawostkę. Jej bezpieczeństwo zatrzymało się 15 lat temu...

  9. Pingback: Huge survey of 6K+ firmware images of devices by 18 vendors including Linksys and Netgear finds … – Trending Topics Worldwide

  10. Pingback: Huge survey of 6K+ firmware images of devices by 18 vendors including Linksys and Netgear finds no improvement in software security and lax security standards (Paul Roberts/The Security Ledger) | Mr Trance Movement

  11. Pingback: Huge survey of 6K+ firmware images of devices by 18 vendors including Linksys and Netgear finds no improvement in software security and lax security standards (Paul Roberts/The Security Ledger) · CYBERDEN

  12. Pingback: Firmware security has barely improved over last decade - Gigarefurb Refurbished Laptops News

  13. Pingback: Firmware security has barely improved over last decade – AB Technology

  14. Pingback: Firmware security has barely improved over last decade – Wilson's Media

  15. Pingback: Firmware beveiliging is nauwelijks verbeterd in de afgelopen tien jaar

  16. Pingback: Firmware security has barely improved over last decade – SahiInformation

  17. Pingback: Firmware security has barely improved over last decade – Nirmallya dey

  18. Pingback: Firmware security has barely improved over last decade - DAILY NEWS -INDIA

  19. Pingback: Firmware security has barely improved over last decade – Maui Municipal Broadband

  20. Pingback: Firmware safety has barely improved over final decade | More Technology Tricks - information technology

  21. Pingback: Firmware security has barely improved over last decade - Fasti News

  22. Pingback: Firmware security has barely improved over last decade – THE 24 HOURS NEWS

  23. Pingback: Firmware security has barely improved over last decade

  24. Pingback: Firmware security has barely improved over last decade - wiredfocus

  25. Pingback: Firmware security has barely improved over last decade • Iphone Paradise

  26. Pingback: Firmware security has barely improved over last decade – Get Free Offer

  27. Pingback: Firmware security has barely improved over last decade | BuyFromtheBrink

  28. Pingback: Firmware security has barely improved over last decade | M&C Recycle Blog

  29. Pingback: Безопасности прошивка не улучшилось за последнее десятилетие читать на сайте AppMaxx

  30. Pingback: Firmware security has barely improved over last decade - Magazine HAMID

  31. Pingback: Firmware security has barely improved over last decade |

  32. Pingback: Firmware security has barely improved over last decade | Berita Terbaru Gen XYZ

  33. Pingback: Firmware security has barely improved over last decade – My Home Tech Reviews

  34. Pingback: Une vaste enquête sur les micrologiciels ne révèle aucun gain en matière de sécurité en 15 ans - Technologik.fr

  35. Pingback: Firmware security has barely improved over last decade - TechnoloJust - News

  36. Pingback: Huge Survey of Firmware Finds No Security Gains In 15 Years - f1tym1

  37. Pingback: Firmware security has barely improved over last decade | Global Trends

  38. Pingback: Firmware security has barely improved over last decade – Sunraynews

  39. Pingback: Firmware security has barely improved over last decade - eGadgetry- Cutting-edge Digital Consumer tech, news and reviews

  40. Pingback: Firmware security has barely improved over last decade - Global-News.Org

  41. Pingback: Firmware security has barely improved over last decade – NEWS 7/24

  42. Pingback: IT Security Weekend Catch Up – August 17, 2019 – BadCyber

  43. Pingback: Huge Survey of Firmware Finds No Security Gains In 15 Years - Papa Mar News Now

  44. Pingback: Links 8/17/19 – Breaking Worldwide News

  45. Pingback: Links 8/17/19 – Viral News Connection

  46. Pingback: Links 8/17/19 - News Views

  47. Pingback: ICYMI: Consumers Should Immediately… Python the Circuit! #Python #Adafruit #CircuitPython #PythonHardware #ICYMI @circuitpython @micropython @ThePSF @Adafruit « Adafruit Industries – Makers, hackers, artists, designers and engineers!

  48. Pingback: CISO View – The Week’s Best News – 2019.08.23 - Mosaic Security Research -

  49. Pingback: 15 Bullet Friday – The Best Security News of the Week – 2019.08.23 - Mosaic Security Research -

  50. Pingback: Wi-Fi 6 Officially Launches Today, With Up to 3x the Performance of 802.11ac - CSITQuestions