Reg called more balanced than Wired

Letter Our recent story on government data mining in Singapore drew a letter form Dave Snowden, Founder and Chief Scientific Officer of Cognitive Edge, which is involved in the project.

Says Dave:

I have just been passed your article in The Register on Singapore's RAHS project, which is slightly more balanced that the Wired article, from which I assume you drew your material.

Just for the record, I agree with more or less everything you say about data mining in terms of its effectiveness when compared to human beings with experience. The aspects of RAHS for which I was responsible assume that humans are the primary interpreters both at the raw data, and large pattern, levels, but also that technology can be used to augment their decision making capability. It's very easy to pick up on the original TIA project in the US (massive data mining) and forget that there was a parallel project (Genoa II), which assumed data mining was not the solution. I worked on that project, and took the ideas across to Singapore's RAHS.

Technology can augment, but not replace, human decision makers. If you looked elsewhere on the blog you would have seen both a response to Wired, and also more recently a summary of the differences between computer and human cognitive processes.

It is to the great credit of the Singapore Government that they have taken an approach based on diversity. Yes, they have data mining (but not TIA) capability from Petersen and others. They also have systems (from ourselves and others) that take a very different approach. Our software is being used (to take one example) to create living oral histories with indigenous people, based on the ideas of emergence (from complexity theory), avoiding crude categorisation or the static nature of many an oral history. It allows people to see patterns, to determine (as I demonstrated in Singapore) nuanced aspects of a government's intent.

So instead of saying "The Iranian Government says X, has Y intent, and will do Z", you represent multiple fragments of public domain material about the Iranian Government as a fitness landscape, in which the unmovable aspects are visible, but the places where there is likely to be an opportunity for change, normally hidden by crude stereotyping, become visible. Such material allows senior decision makers to go from the "big picture" to the "raw data" without disintermediation, and to make better, more nuanced decisions as a result. It also allows them to see the same landscape from different cultural perspectives. At no stage does it attempt to predict a terrorist outrage. That (as you say) is beyond technology in the main.

The knee jerk reaction of Poindexter-TIA-massive data mining-privacy etc. may make for a better story, but it is far from reality.

We won't take issue with you Dave, except over the scurrilous implication that another tech journal could be less biased that this one. ®