Unchaining Documents: The New Data Revolution
The history of technology is about revolution. New tools, new processes and new ways of computing have so altered the way that we live, that it is hard for us to recall life any other way. Have you ever stopped to think about what it must have been like before the Model T made mass car ownership a reality? Can you recall sitting at a typewriter, drafting a memo that would get filed away on a shelf in a massive storage room?
We are in the middle of such a revolution now. The phrase “Big Data” is en vogue right now, but at Lockheed Martin, we see it as more than a fad that is exciting marketers, IT professionals, and government agencies. It is something that is completely altering our operational landscape. And soon, we won’t be able to remember what the world was like before we could truly understand, analyze and leverage Big Data.
The last technology revolution was about the shift from the physical to the digital. But this shift didn’t free up information the way that we had hoped. Information was still trapped in documents. Instead of being fixed on pages and microfiche in file cabinets, data was locked in 1s and 0s on servers.
In fact, the digital revolution actually compounded the problem of information management. Advances in digital technology have turned everyone into a publisher, and gave rise to the phenomenon we refer to as “Big Data” today. Consider the fact that on Twitter alone, it only takes five days for the world to produce a billion tweets. Trying to comb through all of this content, let alone trying to understand it, through manual sorting and analysis is a daunting task.
Whether they are public (like Twitter, Facebook, or other social media posts) or private (like HRIS, ERP, or other business management platforms), our streams of information aren’t designed to talk to one another.
Without a way to help people understand all of these streams of information—not just what they say, but what they mean—Big Data will be nothing more than big noise.
The next technology revolution will be about the shift from the document to the data. In other words, technology professionals need to help organizations unchain information from its source, and automate the analysis of that data.
In the national security community, many intelligence analysts now refer to this as object-based analysis. Most data streams that analysts draw from are segmented from other data sources, and the process of acquiring and integrating the information from diverse sources can be time consuming and inefficient.
For example, our intelligence community customers might need to look at volumes of geospatial maps, videos, and other information sources to find one particular location that is of critical importance. This is equally true in the commercial world, where market analysts are trying to learn more about their customers. What magazines does she subscribe to? What are common brands purchased with his grocery rewards loyalty card? Does she have any children, and how old are they? Has he purchased a new car recently?
“Technology professionals need to help organizations unchain information from its source, and automate the analysis of that data”
Instead of manually trying to retrieve and integrate data, Lockheed Martin and other companies are helping our customers automatically extract relevant data from a wide variety of sources, and combine it to identify key threads. A military target, a consumer segment, or an event requiring further investigation are each treated as objects in this paradigm, and algorithms look across various information streams to find attributes associated with that object.
By using advanced analytics to distill huge volumes of information into a single object, analysts (whether examining a military adversary or the marketplace) spend less time collecting and sorting through known information and more time understanding it to identify gaps and additional areas for investigation.
A good example of this is the Worldwide Integrated Crisis Early Warning System, or W-ICEWS. This system that Lockheed Martin developed collects, digests and evaluates mountains of data from open source platforms, such as social media streams, to help forecast future events. The system uses a collection of social science models and aggregates their output based on the historical accuracy of those models to forecast possible future events, such as incidents of unrest that may lead to social instability. Originally conceived as an exploratory DARPA project, W-ICEWS now has users from several dozen government organizations.
Information distillation doesn’t need to only happen at a global level; Big Data can be very personal as well. The same analytic evaluation can take place at a molecular level in the human body. By understanding biological Big Data outputs from the human body, Big Data tools can help hospitals detect illness faster and with greater accuracy.
Data is abundant, and harnessing data is the new revolution. The trick is getting information to work for us, rather than making us work for information. As was the case with so many innovations that have come before, after we get to the point where the information management is truly automated, we won’t be able to remember what it was like in the world before.