Are there really black swans in the cloud?

Gallopin’ Gertie was the nickname given to the Tacoma Narrows suspension bridge in 1940. 

It was so-called because shortly after being opened the bridge developed the unnerving habit of violently undulating during certain wind conditions.  After 4 months the bridge shook itself to destruction.

Today we know the collapse was caused by Aeroelastic Flutter, a phenomenon caused by the interactions of bridge oscillations and flows of air (wind) under certain conditions.

But it took many years of investigation and analysis by engineers and scientists before a clear picture emerged of what happened.  Today, the effects of the flows are understood, lessons have been learned, and bridges (and buildings) are put together more safely.

At the time of the disaster though, if the term had been around, it is likely that the collapse would have been termed by some interested parties as a  ‘black swan event'. (I suppose that makes me a cynic)

The identification of a black swan event has been summarised as

1. The event is a surprise (to the observer).
2. The event has a major impact.
3. After its first recording, the event is rationalized by hindsight, as if it could have been expected (e.g., the relevant data were available but not accounted for).

However it seems to me that the term ‘black swan’ is sometimes attached to a ‘bad’ event, in an attempt to deflect responsibility or because the event is at odds with the prevailing general wisdom.

Information Age recently published, Black swans in the cloud, an article reflecting on the ‘cloud’ outages of Amazon and Sony [emphasis added by me]:

“…This is precisely the kind of outage that was not supposed to happen in the cloud – Amazon’s highly distributed architecture and multiple data centres were meant to provide sufficient redundancy as to reduce the risk to zero. It can therefore arguably be described as a ‘black swan’, an occurrence that is rare, but all the more disruptive when it does happen as a result….”

And

“…Cyber attacks are by no means a rarity, but for an electronics company with Sony’s reputation to be compromised so effectively was shocking…”

So it seems Information Age was surprised by recent events - I’d like to invest in any company that can ‘reduce the risk to zero’!

It’s unfair to single out Information Age.  I think its fair to say the above perception is a feeling held generally in the tech media (and general media) and in some parts of both business and IT: “Amazon and Sony are big tech companies with huge resources, ‘cloud’ is the next big thing - so how could this happen!?”

But to me these major outages are only the beginning.  During the next few years I expect there to be many other ‘cloud’ outages of varying complexity, impact and cost.

Why?

Simply because IT has yet to adopt the engineering approach used in other industries to understand and optimise flows.

On my old blog I wrote in 2008,

"…If we are to properly audit a company’s reliance on IT it is essential that we have sight of how any third party assets through which the company’s data flows are managed.  And we must clearly understand the controls the third party places upon the assets and the data flows…"

Unfortunately the IT industry doesn't yet have that degree of understanding.

Few companies in the world, on-premise or in ‘the cloud’, can see clearly how data flows through people process and technology.  Unlike engineers and, for example, flows of oil or trains or fuel, with a few exceptions IT cannot see clearly how assets are put together to make today’s dataflow reliant businesses work.

That is why there will continue to be outages in flows of data and why cybersecurity will remain a pressing issue.

And if more organisations, both private and public, use cloud computing the impact of major outages will obviously be multiplied across the economy and society.

In my opinion the level of risk we face at the moment for putting business critical data in the cloud is still too high.  As I concluded in ‘A Brief History of Cloud’ a few years ago

"...it will require asking the business to take a leap of faith to find solid footing in the cloud."

The information technology industry has to face up to its responsibilities.  Peoples lives are becoming so intertwined with flows of data that it is surely only a matter of time before some kind of disaster is caused as a direct result of dataflow outage.

As Gallopin' Gertie demonstrated, when business doesn’t understand or manage flow properly, “we have a problem”.

But there are no mystical ‘black swans’.

When the 2010 Gulf Oil spill happened I never heard anyone saying it was a ‘black swan event’.  Mistakes were made in design and/or operations that caused an interruption to the flow of oil.  It was an engineering problem.

I never heard anyone describe the 2002 Potter’s Bar train crash as a ‘black swan event’.   Mistakes were made in design and/or operations that caused an interruption to the flow of trains.  It was an engineering problem.

When Toyota recalled  250,000 cars in 2011 because of concerns about the how fuel was flowing in its vehicles, was it a ‘black swan event’?  No.  Mistakes were made in design and/or operations that caused a potential problem with the flow of fuel.  It was an engineering problem.

Ascribing the ‘black swan’ to events caused by a lack of business clarity, faulty engineering, poor operational procedure or inadequate design, smacks of complacency. 

There are no black swans in the cloud.