Naturally, computers are really good at data – collecting it, storing it, processing it. And by the same token they are also great at working with metadata. Metadata is not the real content, but all the handy labelling attached to the data.

It’s in computers’ digital DNA to assign context to all that lovely data, which is how our files and directories have names with which we can find them. Along with file names comes useful time and date stamps, to record the date of creation and modification. Or even just the time the file was last opened.

And increasingly every file is tagged with even more data about the data, such as exposure settings and geographical location automatically embedded into all the photos we snap with our phones. The more accurately we can describe each document, spreadsheet, photograph, song and video file, the easier it becomes to find on demand, letting us scoop what’s needed from swelling repositories of data that snowball with every season.

The subject of metadata has been raked over again in the wake of the Snowden revelations, making metadata another buzzword paraded through the news. Our governments routinely claim that they are unable to look into our data – the actual content of our emails and our telephone calls, for instance – since these are legally protected by laws and constitutions.

Let’s ignore the facts now laid out before us which clearly show that GCHQ and NSA do indeed record the entire content of every single email and every fixed-line and mobile telephone call that we make today. And instead turn to the amazing utility of our metadata, which the surveillance agencies are more ready to publicly concede they’re routinely tapping and archiving.

On the face of it, there may seem little harm in monitoring the data about the data, so long as the data content itself remains confidential. Therefore I should be assured that while Three – and Andrews & Arnold and BT Wholesale and Apple and GCHQ and NSA – all know that I sent an instant message to my wife from a recorded IP address, using a noted device, at a location that can be pinpointed to within 10 metres, with a timestamp accurate to decimals of a second, none of the above will know the endearing prose itself. (The final three in the chain can likely read the encrypted content as well, but as I say, we’ll let that fact ride for now.)

The problem is that even without knowing the content of communications, metadata creates a very telling story of who we are, with whom we communicate and what we might be saying. Given a suitable slice of metadata, an adversary can build a detailed picture of all aspect of our lives.

Fin de siècle, fin de la vie privée

The British government passed the Regulation of Investigatory Powers Act in 2000 to legitimise the mass surveillance of communications in transit, ostensibly in the interests of national security. This was a landmark point. A significant part of the newly legislated dragnet will include every British citizen’s communications metadata.

But requests are routinely made through RIPA for reasons very far removed from the act’s original stated purpose of safeguarding us from terrorism, organised crime and paedophiles. There are now familiar stories of local authorities hiding behind RIPA to direct surveillance at individuals suspected of mundane civil offences like dog fouling, fraudulent school place applications and fly tipping.

Those are examples of ordinary people being spied upon under the umbrella protection of anti-terrorism legislation. More troubling is the use of metadata harvested for the government that has been exploited to undermine press freedom – notably to reveal journalists’ confidential sources. The Metropolitan police was caught bypassing necessary court orders when it requisitioned and sifted through the phone records of daily newspaper editors in two recent high-profile cases.

The policemen who were found guilty of falsifying evidence in the Plebgate case had secretly obtained phone records (metadata) with a RIPA request to uncover the embarrasing leak. And in the Chris Huhne speeding trial, a confidential source was exposed through the covert requisition of a newspaper editor’s mobile phone records. In these cases it was not the content of the calls that was turned against the journalists and their source, but simply the metadata records that showed the who, where and when of key phone calls.

In November I was invited to the Open Rights Group Convention at Kings College London, by sponsor AAISP. There I heard about the most disturbing application of metadata so far, for the targeted assassination of suspected militants in Pakistan, Yemen and Somalia. The presentation was by Jennifer Gibson, a US human-rights lawyer working with Reprieve. Gibson campaigns to raise awareness about unlawful CIA drone strikes which also kill bystander civilians.

The sickening truth is now unravelling, that drone strikes have been ordered on the basis of intelligence metadata that has condemned targets without trial or right to demonstrate innocence.

Mistakes that kill

The CIA which runs the US drone program sometimes does not know the name or identity of its targets – just their mobile phone number, which is used to pinpoint the strike. Anyone who has called or been called by that number becomes a suspect, which is believed to be the undoing of cleric Faisal Bin Ali Jaber, outspoken iman decrying al-Qaeda in Yemen. He was killed in a drone strike after reluctantly agreeing to meet militants who wished to speak to their critic.

A court case in the UK was brought by Noor Khan last January, the son of a tribal elder and drone assassination victim in Waziristan. Khan tried to sue the British government for its complicity in providing the ‘locational evidence’ (metadata) used to target his father and 40 others. The case was dropped by the British court, as revelations that GCHQ was supplying deadly intel to the CIA would first have to establish that a war crime had been committed by the US. And the British court did not think the circumstances exceptional enough to open that can of worms.

In case there is any doubt about the power invested in metadata, and its usefulness to the surveillence authorities, we need only turn to the spokesmen for the spooks. Starting with former general counsel of the NSA Stewart Baker, who said ‘metadata absolutely tells you everything about somebody’s life. If you have enough metadata, you don’t need content’.

But the last word goes to General Michael Hayden, former head of the NSA from 1999 to 2005, and Director of the CIA from 2006 to 2009. Speaking at a symposium at John Hopkins University in April last year, Hayden concurred that you can learn all about someone from their metadata trail. He followed with the simple, chilling admission: ‘We kill people based on metadata’.

Remember that when politicians like Barrack Obama assure you ‘that’s all we’re collecting, just the metadata, so you shouldn’t need to worry about it’.