How do we improve software quality and end the epidemic of shoddy, exploitable software harming consumers, communities and businesses? To start, we need to change the way we think and talk about software-based risks.
Imagine this: you go to your local supermarket to pick up some food for a barbecue. Perusing the meat aisle, you grab a package of shrink wrapped, raw sausages. As you put them into your shopping cart, a label on the package catches your eye:
Warning: May Make You Sick!
Flipping the package over, you find a long disclaimer from the supermarket chain on the back. Squinting at the tiny print, this phrase jumps out at you: “We use commercially reasonable efforts to deliver our sausages free of E.coli, Salmonella, and other food-borne bacteria.” Not very reassuring, is it?
Sickened by software? ¯\_(ツ)_/¯
But that’s the norm for software. Individuals, governments, small businesses and even giant corporations that purchase software and services today have few guarantees or assurances about the quality or safety of what they consume. It is no surprise, then, that stories about unwitting software consumers being felled by attacks on- or threats hidden inside the software they download, install and run turn up in our news feeds every single day.
Take last week’s news about the discovery of two, previously unknown (or “zero day”), remotely exploitable flaws in the IOS XE software, an operating system made and distributed by Cisco Systems. By Cisco’s own admission, its customers are under attack by malicious actors who are actively exploiting two previously unknown vulnerabilities in the Web User Interface (Web UI) feature of Cisco’s IOS XE software. These flaws affect tens of thousands of physical and virtual devices that run IOS XE software with the HTTP or HTTPS Server feature enabled. Attackers who successfully exploit both flaws, CVE-2023-20198 and, CVE-2023-20273 can create new user accounts, inject commands with elevated (root) privileges, run arbitrary commands on the device, and so on. In essence: they can own (or pwn) the Cisco device.
If IOS-XE was a story about a Salmonella or E.coli outbreak, it would be a national scandal with blaring local and national news coverage. The public would demand detailed exposes on the causes of the outbreak and make loud demands for action from the highest levels of government. Because it is software, however, the collective response of regulators, policy makers and the general public is a shrug.
In fact, like other software makers, Cisco readily acknowledges the possibility that the software it ships may contain flaws or even malicious components. In fact, that language about using “commercially reasonable efforts” is taken directly from a Cisco End User License Agreement, which states that the company will “use commercially reasonable efforts to deliver Cisco Technology free of Malicious Code.”
Lessons from the (actual) sausage factory
It’s time for that to change – for the nation’s tens of thousands of software “sausage factories” to start operating more like the nation’s hundreds of actual sausage factories. In suggesting that, I don’t mean that the work of creating software applications needs to turn into a repetitive, static, assembly-line. What I mean is that the U.S. federal government, facing a decades-long epidemic of costly and disruptive “software poisonings,” should be as invested in the quality of software being produced, sold and consumed in its jurisdiction as it is in the quality of other products – like food- that are produced, sold and consumed.
Consider this: in addition to the extensive food safety and labeling laws in the U.S., food processors since the mid 1990s have been required, by law, to develop and implement systems of preventive controls designed to improve the safety of their products. These are known as HACCP (Hazard Analysis and Critical Control Points). Among other things, food producers must have written sanitation standard operating procedures; they must test their slaughter houses regularly for microbes to verify that the establishments’ controls for the prevention and removal of contaminants and bacteria are working. Slaughter establishments that produce raw, ground products must also establish pathogen reduction performance standards for Salmonella. Importantly: all these rules are enforced. FSIS issues regular reports of its inspections, as well as enforcement actions taken against food processors, distributors, and retailers. In Q2 of this year, for example, FSIS inspected 39.4 million livestock carcasses for disease or contamination, and condemned just over 53,000 of them.
This system isn’t perfect, of course. Outbreaks of diseases like E.coli and Salmonella still occur – but they’re rare – given the massive quantities of food produced, sold and consumed by U.S. residents each day. Alas, there’s nothing like a HACCP for software, but there should be. After all, in 2023, software runs absolutely everything: from manufacturing lines to automobiles to hospital nursing stations.
SBOMs: thinking beyond software ingredients
There are signs that the U.S. government is moving in that direction. Under President Biden’s 2021 Executive Order on Improving the Nation’s Cybersecurity, there has been a concerted push for more software transparency. That includes requirements for Software Bills of Materials (SBOMs) for vendors selling software to the federal government. But those mandates only apply to federal government contractors, not the larger economy.
And even requirements for SBOMs are just a start – the equivalent to food labels designed to spot tainted or expired ingredients (like Log4j 2.14.1). SBOMs can’t identify and correct flaws in the software supply chain, poor coding practices or compromised development pipelines that lead to the distribution of critical, remotely exploitable flaws – or even compromised software – to thousands or millions of businesses and consumers.
Changing the way we talk (and think) about software flaws
What would it take for our software sausage factories to stop making us sick? To start with, we need to change the way we think and talk about software-based risks and contamination. The current nomenclature of “vulnerabilities,” “CVEs” and “exploitation” keep the problem of shoddy and exploitable software that’s pushed into homes and businesses remote.
When we report that “a zero day vulnerability is under active exploitation,” after all, what drops out? Well, the victims of that exploitation, for one, and the harm done by those attacks to companies, customers and the economy, more broadly. The language and the problem of shoddy software remains abstract – devoid of any notion of responsibility or consequence.
Unlike food-borne outbreaks, there’s also little to no effort to establish a context: to answer the “why” and “how” of these failures. That allows them to continue, unabated, without any sense of urgency or even wrongdoing. If shoddy, vulnerable software is just the norm, we consumers keep our expectations low. Software makers are happy to reward those low expectations with balky wares that leave us vulnerable to compromise. And, with little attention to the “how” or “why,” the application development practices that produce 0days will remain unscrutinized.
Uncle Sam to the rescue?
Going forward, we might use language that foregrounds the issue of poor quality code and the harm caused by the distribution of poor quality or compromised software would help. What if we adopted the language of food safety: couching stories like the IOS-XE flaws as being about individuals and companies “sickened” or “harmed” by a flawed software release?
Even more to the point: what if our government took incidents of software “poisonings” seriously: sending experts out to investigate the circumstances that led to the mass release of flaws like CVE-2023-20198, and -20273 to the public. Arm those inspectors and oversight agencies with new laws and regulations, and they might even hold software makers accountable for their mistakes (a crazy idea, I know) with fines and other required remediations to prevent such incidents from recurring.
If the experiences of other industries – like manufacturing, construction and food processing are any indication – the response to changes like these would be a noticeable reduction in occurrences of critical software flaws, software supply chain compromises. Hand in hand with that would be a reduction in successful attacks and exploitation of homes, businesses, and critical infrastructure as software quality and integrity improved, closing off avenues of attack.
Ultimately, cyber attacks are no more inevitable than food poisonings. Both are preventable with the application and enforcement of standards and rules based on an objective understanding of risks and threats. And – as with food safety – we all will benefit in the end.