What happened to Healthcare.gov?
It's been almost two weeks since Obamacare's federal insurance exchange website went live, was inundated with traffic, went weird, was taken down for maintenance, then came back online still filled with glitches. Why did such a crucial site fail at such a critical moment? And what are the lessons that can be learned?
When President Barack Obama rode to reelection in 2012, his campaign was widely praised for the preternatural talents of its IT team, for the innovative data platform (codename: Narwhal) that helped pinpoint and make the most of every last potential vote.
So why, after making it the centerpiece of his presidency and spending untold time and effort defending it from judicial and legislative challenge, has the administration seen the main website for Obama's signature achievement fail so badly at its moment of truth?
When even the usually-anodyne USA Today is calling the site an "inexcusable mess," there are big problems.
In a laudatory profile in The Atlantic last year, the Obama tech team was described as an "elite" squad of "senior" talent (which for techies meant that "most of them were in their 30s") who came from Silicon Valley blue chips such as Google, Twitter and Facebook. They were called a "dream team of engineers" who "built the software that drove Barack Obama's reelection."
Of course, it was a different bunch of folks who set up the Obamacare website. The chief contract went to CGI Group, a Canadian firm that help set up that country's single-payer sites. Columbia, Md.-based Quality Software Services was also enlisted by the Centers for Medicare & Medicaid Services to build data infrastructure, as were several other contractors.
Recognizing that the federal bureaucratic procurement process operates differently than a lean and creative campaign technology team, it's still worth asking how an ostensibly forward-thinking and IT-savvy administration could get the technology so right in November 2012 and so catastrophically wrong in October 2013. After all, couldn't they see what a big deal this would be? Why weren't they ready?
Well, technology can be complicated. And unpredictable, even with the best planning. As that Atlantic profile put it, even as the 2012 campaign was in its home some stretch, the much-vaunted Obama techies realized their best-laid plans could go astray.
"I know we had the best technology team I've ever worked with, but we didn't know if it would work," the campaign's chief technology officer Harper Reed told the magazine. "I was incredibly confident it would work. … We had time. We had resources. We had done what we thought would work, and it still could have broken. Something could have happened."
The Healthcare.gov debacle has shown that even with time and resources – the Supreme Court upheld Obamacare back in June of 2012; all told, CMS is on the hook for nearly $400 million for the website so far – things do indeed sometimes happen. On Oct. 1 something sure did. What was supposed to be the dawning of a new era turned into a nightmare for the Obama administration as the site wilted under the pressure.
And so it was that Jon Stewart threw down the gauntlet to HHS Secretary Kathleen Sebelius on The Daily Show this week: "We're going to do a challenge: I'm going to try and download every movie ever made, and you are going to try to sign up for Obamacare – and we'll see which happens first."
The website did not recognize some users; for others, it prevented them from setting up accounts; some were asked to confirm email addresses, but couldn't because of faulty links. Within a couple days, the entire site was taken down for maintenance. Even now that it's back online, it's having problems.
Indeed, the issues have been irksome – and potentially damaging to the larger goal of Obamacare. How many of those reported 9 million visitors from 36 states simply decided not to get coverage – decided that it wasn't worth the headaches – never to return?
"The biggest thing we've learned is that, in retail – which, granted, isn't the same case as here – but in retail, if you don't get the user experience right the first time, consumers will go somewhere else," Tom Lounibos, chief executive officer of Mountain View, Calif.-based SOASTA, which specializes in testing high-traffic websites before go-live, tells Healthcare IT News. "Those brands lose revenue if it's not a quality user experience."
So what happened? It wasn't the underreported fact that the website was initially built in a garage (presumably not from duct tape, plywood and fishing line). No, the failure appears to have come from any number of technical causes, coupled with an unfortunate lack of foresight.
As Government Health IT's Anthony Brino showed earlier this week, a crowdsourced group of computer geeks on Reddit have been doing some amateur sleuthing, unearthing bad bits of front-end coding, snafus with Javascript and CSS files and HTML tags, etc.
Experts interviewed by the Wall Street Journal found a "glut of stray software code that served no purpose they could identify."
But as was argued in Slate this earlier this week, the site's "biggest problems are most likely not in the front-end code of the site's Web pages, but in the back-end, server-side code that handles – or doesn't handle – the registration process, which no one can see."
As Reuters put it: "One possible cause of the problems is that hitting 'apply' on HealthCare.gov causes 92 separate files, plug-ins and other mammoth swarms of data to stream between the user's computer and the servers powering the government website."
Lounibos inclined to agree. "In this case, I believe there's a back office where they have to go and verify people's names as individuals," he says. "Sometimes testing is done, but they don't test across all environments, and all components that are servicing a particular website. Components might have been tested, but they might not have brought it all together into a single environment: the back-end and the front-end. It seems like, from what I'm reading, that a lot of the problems have been in the identity checking area."