17 December 2007
When database errors kill
If you have nothing to hide, you have nothing to fear. How often do we hear this from those who would promote surveillance, increased police powers and other forms of social control? But there is a basic flaw at the heart of this thinking: who decides what constitutes ‘hiding’? How do you decide what is acceptable behaviour?
In essence, it comes down to what is considered ‘normal’. And what is unspoken is the assumption that those who do not fit into these categories in some part of their existence - their beliefs, their attitudes, their behaviour, their concerns and even their race - are in some manner guilty by default.
Is being different a crime? Within a totalitarian regime, it is - always. Such regimes demand complete compliance in thought and deed. And increasing demands for such compliance, and the imposition of arbitrary standards of normality, are signs that a society is drifting towards totalitarianism. Sometimes this is religiously inspired. Other times it is the work of a self-serving state.
The problem today is that such notions of ‘normal’ and ‘abnormal’ are being enshrined and encoded in what might appear to be neutral systems. Take surveillance, for example.
Authorities in countries like the US and the UK are deploying surveillance systems that have some degree of ‘intelligence’. New video camera systems analyse the gait - the manner of walking - of people in a crowd. The systems’ computers are able to spot anyone who is acting ‘unusually’. This might indicate, we are told, the presence of a explosive vest under a coat, or just a degree of agitation or guilt.
The Government in the UK wants to introduce road pricing. This involves every vehicle being fitted with a remotely readable identification device. Your every move will then be tracked and logged, and you will receive a bill through the post for the roads you have taken. This is proposed under the guise of an ecological benefit - it will encourage people to use public transport, or share vehicles, or not travel. For certain government agencies, however, it has another benefit. It is believed by many that they will deploy systems that will analyse traffic movements and spot suspicious activity.
There are many other, similar systems that use degrees of artificial intelligence to alert authorities to suspicious activity. But in order for such systems to work, they must have encoded into them a model of what constitutes ‘reasonable’ or ‘normal’ behaviour. This model, then, will inevitably incorporate the prejudices of their programmers (who, almost by definition, will tend to be conservative and authoritarian).
Anyone who does not behave according to these norms is thus subject to suspicion. And in our paranoid world, simply becoming an object of suspicion is dangerous enough. Ask anyone who has been subjected to extraordinary rendition, or who has been imprisoned without charge or trial at Guantanamo Bay.
And that’s assuming the system works as planned. Alas, no system is perfect. Aside from the prejudices and misconceptions that may be present in the design, all complex systems contain faults and are subject to errors. Databases, in particular, always contain faulty data. Any database programmer will tell you this. And in an area that concerns individual privacy and liberty, mistakes may be costly indeed.
To take one instance in the US: Michael A Dodele believed he had nothing to hide. Or, more accurately, that he would not be allowed to hide. He was released from jail after serving 20 years for sexual offences against women. Under ‘Megan’s Law’ (see footnote in comments), details of sex offenders are made publicly available on a web database. But there was a mistake in Dodele’s record that suggested his crimes had been committed against children.
Dodele was stabbed to death in his trailer home. The chief suspect is a man who lived nearby and who claimed that his own son had been molested in the past.
It wasn’t the authorities who killed him - they can wash their hands of the affair. They can point out that whoever killed him was acting unlawfully, that the punishment exacted was not sanctioned by or acceptable to them. Nevertheless, they created a system in which a mistake can have dreadful consequences. And because it’s an official system, it carries the weight of authority.
One may find it hard to have sympathy for a convicted rapist. But imagine this: what if the mistake that was made was not in the description of his offence, but in some other field of the database record. What if the record had been accurate in all details, including the savagery of his crimes, but instead of putting his name and address the authorities had accidentally entered yours?

Amazon.com
Amazon.co.uk


(1) 19 December 2007 at 9:53 am
Steve
Footnote:
One person implicated by Megan’s Law is still alive - the man who killed Megan. Jesse Timmendequas was convicted of murdering Megan Kanka in 1994. He has been on death row in New Jersey ever since, but his life will now be spared. New Jersey has abolished the death penalty.