In this episode of the podcast (#233) Mark Stanislav, a Vice President at the firm Gemini, joins Paul to talk about what went wrong with disclosure of Log4Shell, the critical, remote code execution flaw in the Log4j open source library. Mark talks about how the Internet community can come together ahead of the next vulnerability to make sure the mistakes that are evident in the response to Log4j aren’t repeated.
As always, you can check our full conversation in our latest Security Ledger podcast at Blubrry. You can also listen to it on iTunes and Spotify. Or, check us out on Google Podcasts, Stitcher, Radio Public and more. Also: if you enjoy this podcast, consider signing up to receive it in your email. Just point your web browser to securityledger.com/subscribe to get notified whenever a new podcast is posted.
Back in 2008, the late, great security researcher Dan Kaminsky discovered a serious security flaw in a ubiquitous Internet technology: the domain name system, or DNS. If widely disclosed and exploited, the flaw – which affected many of the most common DNS name servers – could have facilitated a wide range of attacks, including website impersonation, email interception, and authentication bypass hacks.
Aware of the risks, Kaminsky worked quietly for months with the Department of Homeland Security, major tech firms like Cisco and Microsoft as well as DNS providers to get patches written and distributed – all before details of the vulnerability were made public. Vendors worldwide were able to take steps that largely mitigated the risk of attack before any details of the flaw became publicly known.
Log4j Disclosure Chaos
That’s not how it happened this month with another ubiquitous security vulnerability emerged: Log4Shell, a flaw in the open source logging library Log4j that is a common element of thousands of on premises and cloud applications used by enterprises, governments, critical infrastructure operators and individuals.
Rather than coordinated disclosure along the lines of Kaminsky’s DNS flaw, the world experienced something akin to coordinated chaos with Log4j, which first came to light via a patch by the video game maker Mojang Studios to its MineCraft game on December 10. That preceded public acknowledgement by the Apache Foundation, which had quietly issued a patch for the issue on December 4.
Disclosure by Mojang was followed by confirmation of the flaw by other researchers, including one at the firm Alibaba. (The company later faced sanctions by the Chinese government for not disclosing it to them first.) Reports surfaced almost immediately of active scanning of the Internet for applications that were vulnerable to the Log4Shell RCE.
Active Exploits and a Flawed Patch
Complicating matters, Apache’s first attempt at a fix for Log4j, Version 2.15.0 was – itself – flawed and did not fully patch the Log4Shell vulnerability, further confounding efforts to patch. More than two weeks later, organizations are still scrambling to assess their exposure to the Log4Shell vulnerability. Meanwhile, both everything from ransomware gangs to nation states are actively attempting to exploit the hole to gain access to sensitive networks and systems.
It didn’t have to be this way, says our guest this week. Mark Stanislav is a Vice President of Information Security at the firm Gemini. He notes that, while Log4j was destined to be a big deal and require widespread patching and mitigations, the response to the flaw did not need to be as messy as it has become. Long-recognized best practices in coordinated disclosure simply weren’t followed in the case of Log4j – for reasons that aren’t entirely clear. That has had profound repercussions for the security of billions of Internet users.
In this conversation, Mark and I talk about what went wrong with Log4j, the problem of information “entropy,” and how the Internet community can come together ahead of the next vulnerability to make sure the mistakes that are evident in the response to Log4j aren’t repeated.
Listen to the podcast above, or click the button below to download the MP3 version!