Health Exchanges Need A Fail Whale

In a blog post on Veracode’s blog today, I write about the problems encountered at government-run online health exchanges that were intended to connect millions to private insurance plans under the Affordable Care Act. The exchanges opened to the public on Tuesday, and they got off to a rocky start, with reports of web sites paralyzed as millions of uninsured Americans logged on to sign up for subsidized health insurance. Unavailable, the federal government’s main online health exchange, was overloaded by millions of visitors. Was that avoidable? Security experts say “yes.”

In some cases, the problems appear to have been caused by “external factors.” New York State’s online health exchange was felled by the weight of more than 10 million requests of dubious origin, The New York Post reported. But other exchanges, including the federal government’s main health insurance storefront, which is used by residents or more than half of the states, were victims of their own success: overwhelmed when the doors swung open and millions of eager customers poured in.

Surprising? “No,“ according to web application and security experts. “I’d never rule out government incompetence,” Jeremiah Grossman of the security firm WhiteHat Security told me. “But I think if you ask people who build scalable web systems if they would expect to stand up a system and have two million people use it on the first day without any problems, they’d say ‘no way.’”

The problems facing such web-based applications are legion. Sites must have adequate bandwidth to handle the volume of traffic and simultaneous connections that such a spike creates. The backend database must be programmed in a manner that can manage thousands or tens of thousands of simultaneous queries.

Grossman noted that popular sites like Google, Facebook and Twitter all had their share of availability problems. Twitter’s Fail Whale became the world’s most famous “503 error,” and was such a common occurrence that it even garnered media attention for the (Australian) artist who created it. But those sites had months or years to work out the kinks before their user base grew to millions, or tens of millions of users – the situation and other exchanges now find themselves in.

Red White and Blue Fail Whale
The government’s online health exchanges could have used Twitter’s “Fail Whale” this week, as they struggled to keep up with demand.

In other words, “no surprises here.” But, as I write on Veracode’s blog today, the government could have done a better job anticipating demand for its services and making design and deployment decisions that would have avoided the door-buster effect of throwing open access to all the exchanges everywhere all at once.

“These systems should be sized for the population that would likely access them,” Chris Wysopal of Veracode told me. “It shouldn’t be that hard to predict…All of the problems encountered are well-known and could have been avoided.  Of course some of that avoidance costs money so the tradeoff is do you want a great day first day experience and no snafus over time or do you want to save money?”


You can read more over at 

Spread the word!

Comments are closed.