SOD Big Data! Most of what you're keeping is digital landfill

I've been an industry analyst a long time – and I have seen a lot of "the next big things" come and go. In fact our appetite as an industry for the next big thing rivals that of Simon Cowell in pursuit of the next big star. And just as Simon's "stars" are usually anything but, so too are we in the IT sector often deluded, and ultimately find ourselves short-changed.

You might think we would have learned by now, but sadly no, we still tend to buy into the next craze – the next trend that is going to take us to technology nirvana.

I was going to say that the current all-consuming craze is for all things Big Data, but frankly it seems that this latest fashion is already showing some signs of wear. Let's be honest: Big Data just doesn't look as shiny and appealing as it did a year ago. Odd because it promised so much, in fact it promised us the world and more.

But though the virtual echo chamber consisting of vendor, reseller, analyst and press managed to convince itself that Big Data really was the next big thing – buyers saw through the BS.

It's useful - if you store a LOT of info on your customers

It's certainly useful for the likes of Google, Apple, Facebook and LinkedIn, they all have real Big Data issues and opportunities. But the reality is that most of us don't and never will. Our Big Data issues are issues of data mismanagement, a chronic tendency to hoard junk and never clean house.

To quote the great Joan Rivers: "You make the beds, you wash the dishes and then in six months you have to start all over again!"

Up until now large public and private organisations have simply hoarded all data, pulled in by the allure of ever-cheaper storage and the mystical properties of the Cloud. This addiction to hoarding has seemingly been validated by the promise of Big Data and Analytics. It is a destructive, costly and illusory addiction if ever there was one.

Have a good clear-out...and save some dough

But last year something odd but very important happened to challenge this situation, and you would be forgiven for having missed it. IBM stopped talking about the retention of business data, and turned its marketing on its head. Instead of focusing its messaging as it always has done on retaining/holding/keeping data, it started to talk about something called "responsible disposition", posh words for the art of getting rid of stuff and cleaning house.

The result has been that this particular business unit within IBM has seen its sales nearly triple in a remarkably short period of time. For hoarding mountains of junk, redundant, duplicated, incomplete and corrupted data – and hoping that through the miracle of an analytical algorithm we might finally find something of value in the heap is one way of looking at information management. Identifying that same mountain of junk, redundant, duplicated, incomplete and corrupted data as early as possible in its lifecycle and getting rid of it is another, and frankly far more sensible way of managing information.

IT buyers and end users the world over are crying out for some common sense to enter into the world of IT. Some savvy vendors are actually starting to listen to those calls and develop systems and approaches that address those needs. It's a 360° turnaround that is long overdue, but it's welcome nonetheless.

Automated Information Governance, driving the analysis and management of informational data throughout its lifecycle from inception to destruction - keeping only what is good and necessary, and clearing out what is not good or necessary as quickly as possible - has to be the future of information management.

Buyers want it, because they can see an immediate business case, fast return on their investment, reduction in IT spend and gain in operation efficiencies. Some vendors of course will always have a vested interest in taking the opposite approach, for who wants their customers to really buy less capacity and become less reliant on your services? (It's a little like McDonalds selling salads - we know their hearts aren't really in it.)

But really smart vendors can also see that to move from the current situation - of data ruled by the hand of the Great Anarch - to a dramatically slimmed down and improved implementation will take time and money. It will open up many more new and unique opportunities in its wake.

We are moving from a world of reactive information management to a world of pro-active information governance. There are no magic bullets out there, but by simply rethinking the situation and working to break the addiction to data hoarding we can make progress. All data is not equal - most of it is just noise.

Right now there is far more noise than there is signal, and we need to improve that ratio.

Certainly if you are Google, Apple, Facebook or LinkedIn, trawling through all the activity data that your users generate finding connections and attempting to monetise those interactions is core to your business. But for most of us, reining in the digital landfill that eats away at our IT departments and budgets is surely of much greater importance. ®