Stale pizza, backup BlackBerrys, payroll panic: Sony Pictures mega-hack

Sony Pictures has revealed a behind-the-scenes look at how it handled its recent megabreach to select media outlets.

Extensive accounts of the unfolding disaster by the New York Times, Wall Street Journal (here) and elsewhere reveal that Michael Lynton, the studio’s chief executive, communicated with other senior execs using mothballed BlackBerrys previously kept in a storage room in the basement of its Culver City, California headquarters, after regular communications systems were taken out by hackers. Computers on the firm’s network are laid low by a particular vicious outbreak of wiper malware that left the firm without email.

The attack, which hit three days before Thanksgiving in late November, also left the studio without voice mail or production systems.

Updates on the hack were relayed from person to person by text or call.

Technicians, who had begun working around the clock to contain the problem in an office subsequently littered with stale pizza, were debating whether to take Sony Pictures entirely offline. Meanwhile administrators brought out old machines that allowed them to issue physical payroll cheques after computer network problems made regular electronic direct deposit impractical if not impossible.

Despite the extreme disruption, the hacking was viewed as nothing more severe within Sony Pictures than a “colossal annoyance”. It was only when hackers leaked sensitive information that Sony Pictures realised it had to be more proactive and by then the movie studio was heading towards a dual operational and PR disaster.

Sony Pictures was starring in its own disaster movie, along the lines of the Poseidon Adventure, with a supporting cast and crew including FBI investigators, as the NYT explains.

By December 1, a week after Sony discovered the breach, a sense of urgency and horror had penetrated the studio. More than a dozen FBI investigators were setting up shop on the Culver City lot and in a separate Sony facility near the Los Angeles airport called Corporate Pointe, helping Sony deal with one of the worst cyberattacks ever on an American company.

Mountains of documents had been stolen, internal data centers had been wiped clean, and 75 per cent of the servers had been destroyed.

Everything and anything had been taken. Contracts. Salary lists. Film budgets. Medical records. Social Security numbers. Personal emails. Five entire movies, including the yet-to-be-released “Annie.”

Sony Pictures seemingly lacked anything approaching an adequate disaster recovery plan or any incident response capability. There was seemingly no plan B to switch operations to another location in extreme situations. And where were the several backups or backup systems of any kind? The studio is sadly destined to be a case study in what can happen in the absence of disaster recovery and incident response for years to come.

The studio’s handling of the PR shit-storm spawned by the mega-hack is scarcely better. The (subsequently reversed) decision to cancel the planned Christmas Day release for The Interview, the controversial film (that according to the official version, at least) provoked the North Koreans into launching a full-on assault at the studio, was a particular low point on the PR front. Sony was subsequently criticised by both President Obama and Hollywood celebrities for the perception it caved into vague threats invoking 9/11 from the hackers that movie theatres would be attacked if they showed The Interview.

The NYT reports that an email sent on 21 November by by “God’s Apstls” contained a demand for Sony to pay off the hackers before 24 November in order to avoid a more severe attack. The self-style Guardians of Peace hacking crew claimed responsibility for the attack. Internally the idea that North Korean might be involved was “little more than a paranoid whisper” around this time, the NYT adds.

Sony Pictures has sought to portray the attack as unprecedented and nigh-on-impossible to prevent since it was the work of state-sponsored hackers, who used compromised internal credentials and wiper malware to wreak havoc.

Security experts continue to question this interpretation of events as well as the official line that North Korea is to blame. The alternative theory that a disgruntled former employee teamed up with criminally minded hackers or politically motivated hacktivists is gaining currency. ®