Intelligence: How Data Mining Saves Lives

Archives

June 13, 2006: One of the most effective intelligence tools used in combat today, predictive analysis, was invented a century ago with the development of junk mail. Who knew? For decades, the statistical tools used to determine who to send junk mail to (so the sender would make a profit) were not much use to the military. Then came cheaper, and more powerful computers, and the development of data mining and analysis tools. This made a big difference, because the more data you have to work with, the easier it is to predict things. This has been known for centuries.

Now, with thousand dollar laptop computers equipped with 80 gigabyte hard drives, you can put large amounts of data in one place, do the calculations, and make accurate predictions. This wasn't possible thirty years ago, when a 75 megabyte hard drive cost $45,000, and the computer doing the calculations cost even more than that. You also didn't have digital photography (more data you can store for analysis), or a lot of data, in general, stored electronically. It's all different today. That 80 gigabyte hard drive (holding over a thousand times more data than the $45,000 one of yore) costs less than a hundred bucks.

In the last few years, intel analysts have realized how powerful their tools are. And for those who studied math, statistics or business in college, they know the power of data mining, because it has become a very popular business tool. In places like Iraq and Afghanistan, lots of data is being collected all the time. It was data mining that led to the capture of Saddam, and the death of Zarqawi. Actually, over a hundred senior (team leader and up) al Qaeda terrorists have been killed or captured in Iraq using these techniques.

Data mining is basically simple in concept. In any large body of data, you will find patterns. Even if the bad guys are trying to avoid establishing a pattern to their actions, they will anyway. It's human nature, and only the most attentive pros can avoid this trap. Some trends are more reliable than others, but any trend at all can be useful in combat. The predictive analysis carried out with data mining and other analytic tools has saved the lives of hundreds of U.S. troops, by giving them warning of where roadside bombs and ambushes are likely to be, or where the bad guys are hiding out. Similarly, when data was taken off the site of the Zarqawi bombing, it often consisted only of names, addresses and other tidbits. But with the vast databases of names, addresses and such already available, typing in each item began to generate additional information, within minutes. That's why, within hours, the trove of data generated dozens of raids, and even more leads.

Speed has always been an advantage in combat, but, until recently, rarely something intelligence analysis was noted for. No longer. Predictive analysis is something the troops depend on, not only to tips on what to avoid, but for names and places to go after.

 


Article Archive

Intelligence: Current 2018 2017 2016 2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 


X

ad
0
20

Help Keep Us Soaring

We need your help! Our subscription base has slowly been dwindling. We need your help in reversing that trend. We would like to add 20 new subscribers this month.

Each month we count on your subscriptions or contributions. You can support us in the following ways:

  1. Make sure you spread the word about us. Two ways to do that are to like us on Facebook and follow us on Twitter.
  2. Subscribe to our daily newsletter. We’ll send the news to your email box, and you don’t have to come to the site unless you want to read columns or see photos.
  3. You can contribute to the health of StrategyPage. A contribution is not a donation that you can deduct at tax time, but a form of crowdfunding. We store none of your information when you contribute..
Subscribe   Contribute   Close