The Benefits, and Ethical Questions, Surrounding Metadata

The majority of people in the United States are producing and distributing data on a near-constant basis. Some of this is intentional - traditional methods like conversations and writing and newer digital methods that, at last count, were used primarily to distribute pictures of cats and Starbucks orders. But a majority of this data sharing is unintentional or passive.

This unintentionally shared data, alone as individual bits of information, is typically insignificant - the length of a particular phone call, or the temperature outside when a particular purchase was made. But when combined in the right way, these individually insignificant pieces of unintentionally shared data can form a workable data set fit for analysis. This workable set, whether pulled from willingly or unintentionally shared information, is what is known as "metadata" - technically defined as "data from data". 

Compiling and analyzing metadata is becoming a must-have skill and is shaping the future of services and products. Google Maps predicts traffic effectively, and helps users around the U.S. plan their speedy travel routes,  by gathering speed information from the millions of GPS-equipped Android smartphones in traveller's pockets. In a recent article on ThomasNet's Procurement Journal, we discussed how university groups are looking at Twitter patterns and Google searches to generate predictions of unemployment rates months before government studies confirmed their accuracy. 

But the ethical questions surrounding the gathering of some of this data are hard to ignore. While no personal information is collected, the sheer amount of data collected, when paired with innovative and comprehensive analysis techniques, allows analysts to develop profiles that might are as good as what would be revealed through personal disclosures. This is the crux of the NSAs highly publicized, and highly questionable, data collection practices - in which they attempted to leverage large-scale metadata analysis to identify potential terrorists and terrorist plots. 

These ethical questions aside, cities around the country are now beginning to collect and leverage metadata to further improve their infrastructure. This past week, Chicago announced their intention to install sensor networks across their streetlamp network to monitor all manner of environmental and human factors, including pedestrian movements (via cell phones). While this has the potential to do good - adjusting streetlight timings, predicting neighborhood and population shifts, adapting public transportation patterns, etc. - it also has the ability to do harm. 

And these benefits and concerns carryover to organizations wishing to gather and make use of their own metadata. Whether gathered from a well-managed telecom network or a from a thorough analysis of all contracts and invoices, the metadata within your own organization can be helpful or harmful. And it's important to establish policies on data analysis, and quell internal concerns about data gathering, if your organization wishes to effectively implement a collection program that stays within the law and within your employees' good graces.

The use of metadata within the office can do everything from saving money on your telecom spend to effectively diagnosing redundancies and inefficiencies in workflows and processes. However, these benefits are moot if your workplace morale drops, or you cross the line and get sued.