August 10, 2011

Lightning Strikes Twice for Amazon And Microsoft

A major service disruption was experienced by European users of Microsoft and Amazon cloud services this week and Amazon users stateside suffered a cloud outage for popular services like Foursquare, Reddit, and Netflix.

Netflix tweeted that,"We're aware that some members are experiencing issues streaming movies and TV shows." The problems were resolved about two hours later.

The issues stateside happened after Amazon network traffic was "executed incorrectly." Network traffic, instead of being shifted to another router, went to a lower-capacity network, taking down servers in Northern Virginia.

Also a power utility provider in Dublin, Ireland was struck by lightning, causing an explosion that took down the back-up systems for the region.

Amazon told its users through its service health dashboard that a power outage would usually be "picked up seamlessly by backup generators", but in this case, "The transient electric deviation caused by the explosion was large enough that it propagated to a portion of the phase control system that synchronizes the backup generator plant, disabling some of them."

These power sources needed to be "phase synchronized" manually, causing delays to the resumption of Amazon's EC2 (Elastic Computing Cloud) and RDS (Relational Database Service).

Amazon said, "Due to the scale of the power disruption, a large number of EBS servers lost power and require manual operations before volumes can be restored. Restoring these volumes requires that we make an extra copy of all data, which has consumed most spare capacity and slowed our recovery process." Amazon should complete their recovery within the next 24 to 48 hours due to added capacity.

The same lightning strike created a cloud service outage for Microsoft, who is coming under fire for outages to their BPOS. They said that the outage caused connectivity issues for European BPOS customers for nearly seven hours.

Microsoft has come under criticism for the number of service interruptions to BPOS.


On the Net: