In the last few years, similar glitches from companies like Amazon have temporarily shut down systems across the globe, and this latest issue comes as a result of a botched software update from cybersecurity firm Crowdstrike, whose link to mega customer Microsoft has led to worldwide problems — including chaos in airports, stock exchanges, and hospitals, though a fix has now been deployed.
This time the scale is unprecedented. That should spur Microsoft and other IT firms to do more than simply administer a band-aid. Policy makers could address the world’s over-reliance on just three cloud providers too.
Today’s reality, where a single bug can harm millions of people at once, doesn’t have to be the status quo.
Network technicians and engineers have been scrambling to address the ‘blue screen of death’ that has popped up on Windows computers around the world, effectively making them useless. It’s forced airlines to write their flight times on whiteboards and issue hand-written paper tickets; British TV channel Sky News was forced to go off the air.
The glitch is due to an update of Crowdstrike’s Falcon software, ironically designed to prevent harm from viruses and cyber threats and described as a “tiny, single, lightweight sensor”.
Falcon counts Microsoft as a key customer and crucially, has privileged access to one of the most fundamental cores of an operating system like Windows, known as the kernel.
In theory, this is a good idea. If Crowdstrike’s tool didn’t have this access, then any malicious hacker who got root access could simply deactivate Crowdstrike’s antivirus software and run rampant.
But it’s now obvious there’s a flip side to having that kind of privileged access, if Crowdstrike itself makes an error. That’s why blame shouldn’t just fall on Crowdstrike (whose shares fell by more than 20% early yesterday) but also on Microsoft for arguably not designing a more resilient operating system.
Damningly, Apple and Linux’s operating systems were not impacted by the glitch at all, according to a blog post from Crowdstrike on Friday. And neither appears to give Falcon such privileged access to their kernel, which now looks unwise. Microsoft didn’t respond to a request for comment.
This wasn’t a cyberattack but, like previous outages, the result of the Byzantine complexity of cloud IT processes. The cybersecurity industry has done a stellar job in the last decade in marketing itself as a salve to all manner of frightening threat actors, but one downside may be that companies have neglected basic IT hygiene as that infrastructure becomes more intricate.
Earlier this year, Palo Alto Networks chief executive Nikesh Arora said:
One technical solution might go back, naturally enough, to the age-old trick of “turning it off and on again” — restarting a system twice when updating software. The first boot applies the update, and the second makes sure the system is stable before fully activating the changes.
But these are only piecemeal solutions. The bigger problem is the supply chain itself for cloud computing and, by extension, cybersecurity services, which has left too many organisations vulnerable to a single point of failure. When just three companies — Microsoft, Amazon, and Google — dominate the market for cloud computing, one minor incident can have global ramifications.
EU countries are furthest ahead in addressing the market stranglehold that these so-called hyperscalers have with the new EU Data Act, which aims to lower the cost of switching between cloud providers and improve interoperability.
US legislators should get in the game too. One idea might be to force companies in critical sectors like healthcare, finance, transportation, and energy to use more than just one cloud provider for their core infrastructure, which tends to be the status quo.
Instead, a new regulation could force them to use at least two independent providers for their core operations, or at least ensure that no single provider accounts for more than about two-thirds of their critical IT infrastructure. If one provider has a catastrophic failure, the other can keep things running.
As painful as the outage has been, it’d be a waste to not use it as a catalyst to stop what is fast becoming a recurring nightmare.
- Parmy Olson is a Bloomberg Opinion columnist covering technology
