It has been almost a year since security researcher Dan Farmer first warned of the danger posed by Intelligent Platform Management Interface (IPMI) – a ubiquitous protocol used to do remote management of servers. According to a new report, however, that warning went unheeded.
Writing last week (PDF), Farmer said that a world-wide scan for systems using the Intelligent Platform Management Interface (IPMI) protocol identified over 230,000 Baseboard Management Controllers (BMCs) exposed to the Internet. As many as 90% of the exposed systems could be compromised by exploiting what Farmer characterized as “basic configuration and protocol weaknesses.” Even more worrying, the 230,000 systems that are Internet accessible are probably just a fraction of all the vulnerable systems that might be attacked, with many deployed on (hackable) corporate and private networks.
Farmer is reiterating calls for public and private sector organizations to wake up to the dangers posed by IPMI. Hackers who are able to compromise Baseboard Management Controllers (BMCs) – the small, embedded systems attached to a system’s motherboard that manage IPMI communications, would easily be able to compromise other hosts on the same network, he said.
Hardware makers including Dell, HP and Intel, are to blame for the glaring security holes around IPMI. “For over a decade major server manufacturers have harmed their customers by shipping servers that are vulnerable by default, with a management protocol that is insecure by design, and with little to no documentation about how to make things better,” he wrote. “
This isn’t the first time Farmer has warned about the dangers posed by IPMI. In July, 2013, Farmer penned a similar critique, titled “Freight Train to Hell” in which he raised many of the same arguments.
According to Farmer, there are three classes of serious security problems that are common to IPMI:
The first concerns Intel’s IPMI specification, which Farmer says contains flaws that allow intruders to achieve system level access to IPMI-enabled systems without having to enter a password. The Intel specification also calls for unencrypted storage of passwords and allows would-be intruders to access passwords remotely – a dangerous combination.
The second area of concern is with vendors’ implementation of the IPMI protocol and the BMC. Farmer notes that vendors like Dell, HP and IBM have all rolled their own version of IPMI based on the Intel specification. In doing so, most have enabled what Farmer calls “the most insecure features of IPMI” by default, while adding their own features to it – many of them poorly documented or understood. “Most vendors put semi-secret backdoors in their implementations so that their field and support specialists may gain access and control that you cannot.
The third area where problems arise is with how IPMI is used in once IPMI capable systems are deployed. Users often make the most of IPMI’s flexibility: using it to manage servers in very large collections that all share the same IPMI password. “Groups of 100,000 servers or more that share a common password is not unusual with large organizations,” Farmer notes. And, because IPMI is difficult to manage, those passwords often remain unchanged a very long time, he said.
“While none of them are individually showstoppers when combined they create a monumental problem about as large as the Grand Canyon,” Farmer wrote last year.
In his latest research, Farmer and Rapid 7 Chief Researcher HD Moore put hard numbers to the mostly theoretic warnings Farmer made in his “Express Train” paper. The two used Moore’s Metasploit tool to canvas hundreds of thousands of devices running IPMI that were discoverable online. The two found that BMCs running IPMI Version 1.5 were configured so that all accounts could be logged into without authentication. Furthermore virtually all BMCs also had the NULL user enabled, meaning that many systems running the older verison of IPMI could be logged into without either a username or password.
In systems running the latest version of IPMI – version 2.0, the situation is only modestly improved – if that.
Farmer notes that the majority of servers running IPMI 2.0 used “Cipher 0” – an empty encryption protocol that allows users who want to log in to do so without any authentication checks. Users are prompted to enter a password, but any password they enter will be accepted: something Farmer considers “a step back from even the minimal security offered in 1.5.” Of 124,000 BMCs that Farmer and Moore scanned and tested, they estimate around 60 percent were using Cipher 0 and were vulnerable to compromise.
As he did a year ago, Farmer is calling on the vendors that sell hardware that supports IPMI to take steps to address the security holes.
“I personally think it’d be nice to think that our vendors aren’t knowingly driving knives in our backs or otherwise sabotage our possibly futile efforts to secure our systems as we sign checks to them.”
Warnings about the security of embedded systems are growing louder. Speaking at the recent Security of Things Conference, the Chief Security Officer of In-Q-Tel, Dan Geer, said that vulnerable embedded components like BMCs are the new front line in security.
“An advanced persistent threat…is easier in an environment where much of the computing is done by devices that are deaf and mute once installed or where those devices operate at the very bottom of the software stack,” Geer warned.