As PayPal Returns to Normal, Outage Leaves Questions in Its Wake

As PayPal Inc. recovers from a major?and apparently unprecedented?system outage on Monday, questions are starting to emerge about the actual impact on online merchants and about the e-commerce processor's preparedness for such emergencies. PayPal reported late in the day that its network was functioning normally again after the failure of an unidentified piece of network hardware in a PayPal data center interfered with the ability to send or receive payments worldwide. The processor's PayFlow gateway, which many small merchants rely on to process card transactions, was not affected. The outage, which began mid-day, was severe for about an hour and then intermittent for another three hours as technicians scrambled to fix the problem. But estimates of just how many users or transactions were affected are hard to come by. A PayPal spokesperson says by e-mail that the San Jose, Calif.-based eBay Inc. unit doesn't disclose such information. The scope of the trouble, however, was apparently far beyond anything PayPal has sustained before in its 11-year history. “We have had small issues with network hardware in the past, which have impacted certain features of the site a few minutes, but not for the timeframe that we experienced [Monday],” the spokesperson says. The worst interruption in service at PayPal before Monday's outage occurred in 2004 when a glitch in a software update prevented some customers from getting access to their accounts over a span of five days. Ebay reported last month that PayPal processed 259.6 million transactions in the three months through June 30, a 23% increase over the second quarter of 2008. At that rate, the outage, at least in the first hour, could have potentially impacted about 120,000 transactions, though this estimate assumes an even flow of volume each hour. In dollar terms, the processor handles about $7.7 million in payments per hour. The actual impact on merchants may have varied depending on the systems they use for e-commerce traffic, says Scot Wingo, chief executive of Morrisville, N.C.-based ChannelAdvisor Corp., a provider of e-commerce software. “We've supported PayPal for seven-plus years now as a payment system, so through the years have 'wrapped' it with lots of systems that essentially add enterprise-level bulletproofing, queuing, and retrying,” he tells Digital Transactions News in an e-mail message. As a result, he says, the impact for Channel Advisor clients was under 5% of PayPal volume. “However, had we not had that system in place (and most merchants do not), it would have been an hour of lost transactions and two to three hours of sporadic processing,” he adds. “At worst, a bad buyer experience, at worst a situation with lost orders and payments.” Some observers are frustrated by the sketchiness of information from PayPal about the nature and impact of the outage. “I think they need to do more to explain how this happened, how one piece of failed network hardware could somehow bring down the whole network,” says Aaron McPherson, practice director for financial services at IDC Financial Insights, a Framingham, Mass.-based consulting firm, in an e-mail message. Part of that explanation, he notes, should include how the outage could have gone on so long. Entries on the PayPal blog on Monday from a spokesperson named Anuj Nayar gave brief status updates but said little beyond confirming the existence of the problem and noting that PayPal had “all hands on deck” in an effort to fix the issue. The entries also apologized to users for “all the inconvenience.” Nayar told the Wall Street Journal that PayPal is “looking into how to address our affected merchants.” The ultimate impact on merchant trust in PayPal may also be hard to measure. But observers like McPherson say the processor can bounce back quickly. “This has to hurt them with their merchants, but I think they can recover if they do a convincing job of showing how this cannot happen again,” he says.