December 24,2008:
For over two decades, the U.S. FBI (Federal Bureau of Investigation) has
tried to computerize its operations. Billions have been spent, with only spotty
success. While many in the FBI are eager users of computers, and early adopters
of PC technology, there is an institutional resistance, in the organization, to
widespread use of information technology. The FBI is currently in the midst of
its third major effort to computerize the entire organization. This was
supposed to be finished by next year, but now appears to be over budget by $25
million, and delayed by at least a year.
But that's
not the only problem. Congress has regularly stymied the FBI from using
technology that is commonly used by corporations, because of the fear that the
FBI might abuse it. Case in point is data mining. Politicians tend to oppose
this technique because access to too much information creates the possibility
of the government oppressing people. Actually, politicians are well aware that
data mining is more likely to uncover corrupt practices by politicians, than to
cause any problems with the average voter.
Earlier this
year, the U.S. Congress refused to give the FBI $11 million to expand the use
of data mining in counter-terrorism efforts. American politicians are generally
hostile to government use of data mining, a technique widely used, for decades,
in business (marketing), law enforcement (catching criminals) and the military
(finding the enemy). This last use has become much more sophisticated since the
U.S. Department of Defense began pouring billions of dollars a year into
finding ways to defeat IEDs (improvised explosive devices, usually roadside
bombs). The effort to lower IED casualties has opened up all sorts of
opportunities for technological innovation. No one harasses researchers for
using data mining, or makes fun of building supercomputers with graphics
processors (often the same ones found in video game consoles, making super-fast
computers cheap enough to be used in a combat zone to make life saving
predictions), when it saves troops from getting killed.
The data
mining was initially used to figure out who the bomb making crews were, and
where they operated from. Then, using math techniques first developed during
World War II, the intel geeks began creating predictions about where IEDs were
most likely to show up next. These predictive models get better as the quality
of the information going into them improves. As more terrorists are captured
and interrogated, and their computers and data is translated, the predictions
become more accurate.
Using more
primitive computers, Germany employed data mining successfully in the 1970s, to
find leftist, middle class terrorists who were operating with assistance from
the East German secret police. The terrorists thought they were well concealed,
but data mining can do wonders with the slightest pieces of information.
The FBI has
been unable to make this point to Congress, mainly because some key legislators
are ideologically opposed to data mining, and refuse to acknowledge the
widespread success of the technique in civilians and military sectors.