When will the #TeamSnowden newspapers admit to using the same spying tools as the spooks at the NSA and GCHQ?

Power – wielded by technology giants, NSA spooks or media surveillance specialists with sights trained on us — can be addictive
– ‘Willy Bonkers,’ Marzia Faggin, 2013

John Naughton

We offer, in this entry, links that will let readers draw their own conclusions about our belief that the Guardian and New York Times should soon supply full disclosures of their own use of Hadoop, the ‘spying tool’ that the UK and US secret services have been using in so-called ‘mass surveillance’. They must do no less if they wish to hang on to their reputations as great newspapers.

To put the Snowden leaks in proper context, the reporting and editorialising on them should have been shaped by John Naughton – who is not only the technology columnist for the Guardian’s sister-newspaper, The Observer, but an electrical engineer, vice-president of a Cambridge college, and emeritus professor for ‘the public understanding of technology’ at Britain’s Open University. As long ago as last July, he had a small blue fit about media coverage of the Snowden saga – in a column well worth reading, beyond these extracts:

Repeat after me: Edward Snowden is not the story. The story is what he has revealed about the hidden wiring of our networked world. […] This insight seems to have escaped most of the world’s mainstream media […] The obvious explanations are: incorrigible ignorance; the imperative to personalise stories; or gullibility in swallowing US government spin, which brands Snowden as a spy rather than a whistleblower. […] No US-based internet company can be trusted to protect our privacy or data. The fact is that Google, Facebook, Yahoo, Amazon, Apple and Microsoft are all integral components of the US cyber-surveillance system.

We discovered his six month-old protest in a happy accident in which it came up with search terms we used in searching for our last post-Gutenberg entry. Reader, our eyes popped. How, we wondered, had The Guardian not merely buried what Naughton had to say – full fathom five — but marched on with personalising and puffing up the story to such a degree that …

• the technology giants got left out entirely as inventors, enablers and co-operators in surveillance

• newspapers using the identical surveillance techniques conveniently hid this fact behind the Snowden uproar they manufactured

How did The Guardian justify to its conscience avoiding any mention in editorials or reports on Snowden/NSA/GCHQ that in 2011, it actually gave Hadoop, the most powerful surveillance tool – the subject of last week’s post here – a coveted techie award? Curious readers should look up this record in that newspaper’s archive of technology coverage:

‘Apache Hadoop takes top prize at Media Guardian Innovation Awards’

How was conferring that honour explained by the Guardian reporter, Marie Winckler? She quoted one of the software architects responsible for Hadoop: ‘Apache Hadoop pushes data management forward by empowering enterprises to make sense of their increasingly large and diverse collections of data.’ One of the first commenters on this asked sardonically, ‘So what does it do, translated from corporate-speak?’

Ah! He could have found the answer in a 2009 story in the technology section of The New York Times – also a heavy user of Hadoop, and The Guardian’s publishing partner and co-generator of the Snowden hullabaloo. These sections of the piece explained why newspapers find Hadoop so handy for studying their readers, when we click on their sites:

The core concepts behind the software were nurtured at Google.

[…]

[Hadoop] opened the possibility of asking a question about Google’s data — like what did all the people search for before they searched for BMW — and it began ascertaining more and more about the relationships between groups of Web sites, pictures and documents.

Some readers will be scratching their heads, by this point – thinking, but isn’t uncovering patterns and interconnections like these in our web searches and private behaviour  on the net the reason why Edward Snowden sentenced UK and US government spooks to the naughty corner? Yes indeed. This 2009 article in the computer magazine Infoworld answers a few more questions:

‘What’s the New York Times doing with Hadoop?’: A Times software engineer talks about how Hadoop is driving business innovation at the newspaper and Web site

Just innovating for commerce, then, nothing so disgusting as spying … Well, not exactly. Here is what the incredulous reader must study next – even if the verbing of the noun ‘surveillance’ brings on an attack of hives :

‘How The Guardian is Quietly and Repeatedly Spying on You‘

It was almost shocking when I first installed a browser add-on called Ghostery and began to click on various articles at The Guardian. With each click, I discovered that this news publication, which has been primarily tasked with reporting on Edward Snowden and top secret surveillance operations conducted by the National Security Agency, has been surveilling its own readers.

[T]hese publications, while taking on the pious, sanctimonious role of privacy purists, are using multiple third party resources to collect detailed information about nearly every visitor who reads one of the various posts about how the use of digital technology should be a completely private affair. … [ … continues …]