From DNA to Twitter: Data's digital journey to commodity

Big Data and All That In attempting any technological history, one of the traditional difficulties is to know where to begin. This difficulty follows from a flawed assumption that there is a story that is linear and began in one place at one time.

The real world is not like that. Technology is more like a river that has its origins in any number of streams and tributaries, which did not necessarily know they were contributing to the same result.

Technology that works delivers an advantage and traditionally was embraced by the holders of power; financial, political or religious, in order to secure their position.

Knowledge is power, from an excess of which traditionally has leached toxins in the form of abuse, corruption and misery. The key difference between information technology and other technologies is that ultimately the holders of excess power were disadvantaged. It is not possible to grasp the true significance of information technology without considering that context.

Whilst the engineering behind the internet is interesting, what is more arresting is that the internet finally and irrevocably liberated information from the control of those having power and into everyone’s hands. That dramatic and disruptive power base shift for the greater good is the lasting legacy of IT. On account of its abstract nature, a concise definition of information is surprisingly difficult.

Some cosmic physicists may take issue, but I am of the view that in order for information to exist there must be life, and vice versa. Sentience is required in order to find meaning. Only a form of life can evolve to be sentient and act on information, whereas without information transfer, there would be no life as reproduction is not possible.

DNA – not just genetic material after all

DNA is essentially information that can be replicated in order to pass on life to the next generation. The replication is not and must not be perfect. What in IT we might call a bit error is in life what we call a mutation. Mutations cause some variation in the next generation and the environment ruthlessly selects the most successful.

Evolution depends on mutation, without which homo so-called sapiens and other life forms would not have ascended to the point where types of information over and above DNA could be meaningful. Once a life form becomes sentient, natural selection ensures that the senses, such as hearing and sight, which better ensure survival, become more acute. When the senses act upon information received in such a way as to preserve the species, then there is consciousness, a soul, a spirit. The senses form a model of the world of which the sentient being is a part.

As the cognitive powers of homo self-styled sapiens evolved, then curiosity emerged. One description of information is that which satisfies curiosity, but therein lay a trap, because curiosity can be, indeed was and still is, satisfied with the likes of myth, legend propaganda and imagination.

So this definition of information is lacking and instead it is the definition of data and, as will be seen, like a house and a home, the two are significantly different. Some houses are homes; some data convey information. Part of the difference is found in the degree of rigour that applies to the interpretation of data, and of course rigour must evolve later than curiosity in the same way that legislation always lags technology.

What in IT we might call a bit error is in life what we call a mutation. Mutations cause some variation in the next generation and the environment ruthlessly selects the most successful.

The evolution of curiosity prior to rigour resulted in mythology, which can only satisfy the curious if they lack better information. Mermaids, unicorns, centaurs, dragons, Heaven and Hell are testimony to the evolution of story-telling at a time when they were plausible for lack of any better knowledge. The necessary conditions for rigour did not exist.

In the replacement of naïveté by rigour, there is a parallel between an emerging civilisation realising that centaurs are mythical and a child growing to realise there is no Father Christmas. The rigour has demoted the information to data.

Today we find it amusing that people used to buy coloured water from quack doctors at travelling fairs believing them to cure all manner of ailments. Future generations may find it amusing that we bought extremely expensive loudspeaker cables that made no difference to the sound whatsoever.

In order to appreciate that evolution is taking place, it is necessary to be able to make records. Only since records began could we begin to know that now is different to then. Records are information, comparison between then and now is information processing, which must be based on logic. Science is only an evolving repository of information about the universe gathered using rigour.

By the nature of its topic, science must evolve and for that very reason, science thrives on and requires that its present assumptions are challenged. Challenges that succeed refine science; for example, the work of Einstein refined Newton’s Laws to make them accurate under more conditions. Challenges that fail make science stronger. Science and information go together like life and information. Scientific knowledge and the progress of information technology mutually accelerated one another.