Google in 24-hour cloud brownout
App Engine task queue tasks sloo...ooww for ~10% of instances
Google is suffering a cloud cockup, and hasn't been able to fix it for more than 24 hours.
At 14:28 PST on December 2nd, the web giant reported an incident that meant “task queue processing for Google App Engine applications is slower than expected.”
At the time of that incident report, Google said “current data indicates that between 1% and 10% or applications are affected by this issue. For everyone who is affected, we apologize for any inconvenience you may be experiencing.”
More than 24 hours later, at 15:58 PST on December 3rd, Google said “We are still investigating the issue with slow execution of Google App Engine task queue tasks.” In-between those two posts, the Alphabet subsidiary said it was “implementing multiple mitigation strategies to address the slow execution of task queue tasks.”
Google's cloud has behaved reasonably well of late, especially when it runs without manual control. When it has hiccuped, its spasms have generally been brief. At 25 hours and counting, this one looks rather serious, not least because it is not confined to a particular region. We'll update this story as a proper diagnosis becomes available. ®