January 13, 2010: Complaints about the effectiveness of American intelligence have, over the last decade, included unflattering comparisons with commercial intelligence firms, that often do the same work the CIA does. That is, the commercial firms analyze foreign nations, and the probability of those nations prospering, or declining. The latter can mean civil disorder, or even civil war (and definitely a bad place for an American company to invest). Even the CIA buys analyses from the commercial firms, yet the CIA continues to get it wrong with much of the "product" (reports) they deliver to the U.S. government. Unflattering comparisons are made between how the CIA, and their commercial counterparts, operate. The commercial firms are competitive, especially internally. Analysts sign their reports, and are held accountable for accuracy of insight and predictions. Not so in the CIA, where the reports are anonymous (to outsiders), and no one ever seems to get fired for getting it wrong. But the CIA has good reason for operating this way.

The main customer of the Central Intelligence Agency (CIA) is the White House, but it is also supposed to keep the Department of Defense, and everyone else who works for the president, supplied with accurate and up-to-date analysis of what’s going on in the world. But when the CIA analysts present information that does not conform to what people in the White House want to see, there is pressure to modify the conclusions. During the Cold War, the CIA developed a method to deal with this intellectual dishonesty. First, the agency would find out what, if any, outcome the customer wanted. If that contradicted what the CIA had discovered, the report would be modified to please the customer. But the CIA's real conclusions would often be relegated to footnotes, so that the truth would at least remain on record. Often, the best analysts were put on the "B Team" that was hunting for the truth, but could only express it in the footnotes (where "on the other hand" information ended up.) Nobody reads footnotes, at least not at the highest levels of the U.S. government.

This causes problems with all the other intelligence agencies. The director of the CIA is also the DCI (Director of Central Intelligence), which is supposed to mean the “intelligence czar.” Doesn’t work out like that because, as new intelligence agencies grew, or were created, over the last half century, the DCI did not control their budgets. In Washington, you really only control an organization if you control its budget. The other agencies were often appalled at the conclusions the CIA reached, but they also knew how the game was being played, and played along themselves.

All this was great for journalists and historians, who would eventually find the truth, in the form of raw data, and other evidence that backed up the footnotes, and left the main text looking like something an opportunistic politician invented. But, by then, it no longer mattered. This cycle has repeated several times over the last half century, and nothing has changed.

But the emergence of the Internet in the last two decades has made one fundamental change. There is more information available to more people. The intelligence agencies and the media no longer have a monopoly on international information gathering networks. Email and the sheer breadth of data available, all the time, from all over the world, makes it possible for smaller organizations, or individuals, to collect information from all over, and perform their own analysis. People are reaching conclusions that are at odds with the intelligence agencies, and the media. But not with the commercial intelligence firms, who live or die by the accuracy of the analyses they provide their corporate customers.