The CIA recently admitted that it had been on the receiving end of an espionage XX (double cross) campaign during the Cold War. This term first became known to the public in the early 1970s when Britain revealed how they had successfully "turned" every intelligence agent that the Germans had managed to infiltrate into Great Britain, so that for virtually the entire war they were able to feed misleading information to the enemy. Britain called this the XX (double cross) System and it was accomplished with superior operatives and techniques.
The Cold War XX occurred mainly in Europe where the Russians set up efficient intelligence organizations in all the East European nations they took over (as “satellite nations”) after World War II. These mini-KGBs (the ruthless and efficient Soviet secret police) were there mainly to keep the local population under control. That meant it was a very hostile environment for foreign spies. The U.S. paid lip service to these formidable KGB capabilities if only because those who formed the CIA after World War II knew of the British XX System. But over the decades CIA leadership came to believe they were somehow immune to being victims of a Russian XX play. This got worse after the Cold War ended when the extent of how the CIA got played by Russian XX efforts in East Europe slowly emerged. The Russians, like the British after World War II, kept a lot of their XX successes secret for as long as they could. When the XX debacle became common knowledge in the government CIA leaders shrugged it off. But not everyone in the CIA agreed that the XX disaster could just be ignored. A lot of people who worked for the CIA (Americans and foreigners) died because of the XX defeat. The false information the Russians passed on hurt the United States and that mattered. The poor condition of American espionage capabilities mattered. Ignoring the threat in the future increased the risks of getting bit by a XX play again.
The revival of the Cold War with Russia in 2014 (as a result of Russian aggression against Ukraine and American allies in East Europe) has had some interesting side effects. One is a growing rebellion within the CIA against the Cold War mentality that continues to tolerate superior Russian ability to plant and operate spies within the United States while frequently succeeding in turning (into double agents) spies the United States believes it has operating inside Russia. Many CIA officials believe the U.S. is already falling victim to another XX campaign. This problem rarely gets much mention in the media. There are a few exceptions. In 2013 a CIA agent was expelled from Russia after getting caught while trying to carry out an embarrassingly amateur operation. The man (operating as a junior State Department employee at the U.S. embassy) had been arrested earlier while trying to recruit a senior Russian security official. That effort backfired and the CIA man did not detect a trap. Russia later revealed that this also happened a few months earlier but was kept quiet so as to maintain good relations with the U.S. But after the second incident the Russians saw an opportunity to use these sloppy CIA operations for domestic propaganda, to remind Russians that in one area at least they are better than the Americans. To add insult to injury, the Russians also showed their displeasure by revealing the name of the senior CIA official (the “resident” in spy-speak) in Russia. This fellow was forced to leave, which is what the Russians apparently wanted. They hold the CIA resident responsible for this sloppy and embarrassing use of spy craft. By mutual consent, Russia and the U.S. usually keep the names of their own and the other nation’s residents secret. That rule is only breached when you want to send an important message to the other side. Not mentioned was the fact that this sort of sloppiness is usually indicative that XX is in play.
Some say that 2013 incident triggered the increased pressure by active and retired CIA field operatives (the people who recruit and “run” spies in foreign countries) on CIA leadership, and key members of Congress (the ones who have oversight on intel operations) to put an end to the policy of tolerating this poor performance. The field agents point out that when operating against Islamic terrorist groups the old lackadaisical Cold War attitudes (because “no one, at least few Americans, got killed”) is no longer justifiable. It turns out that this policy of sloppy spying and ignoring the effects is alive and well in the CIA and the U.S. government.
The real embarrassment is that, since the 1970s, U.S. spies have become rare and competent ones even more scarce. Normally this does not bother the CIA or the U.S. government which got used to a situation many in the intel profession are coming to regret more and more. But it is a problem when there is an emergency. So, since September 11, 2001, the U.S. has been forced to rely more on contractors, but even skilled freelancers are reluctant to take on the Russians, who have dominated the traditional spy business for most of the last century. If nothing else this new injection of talent, and temporary Russian weakness after the KGB was dismantled after the Soviet Union dissolved, revealed how vulnerable the U.S. really was during the Cold War.
The decline of American espionage competence is an aftereffect of the Church Committee. This was an investigative operation sponsored by Congress in the late 1970s that sought to reform and punish the CIA. The reforms were mainly about eliminating CIA spying inside the United States, or doing stuff for the president that Congress did not approve of. There was also a desire to avoid any CIA connection with foreign unpleasantness (like using unsavory people as spies or informants, paying foreign politicians for information, or using contractors to run informant networks). This led to a growing list of restrictions on what the CIA could do overseas and at home. Congress was out to make sure no future president (the CIA works for the president) could use the CIA, as had been done during the Vietnam War and before.
The CIA interpreted all this as "no more James Bond stuff." After the 1970s, the CIA relied more on spy satellites and other electronic monitoring for their reports on what was going on in the world. But defecting Russians and those from other communist nations were welcome and, all too often, believed when they should not have been. The Church Committee insured that the CIA became a much less interesting place to work for practitioners of traditional (on the ground, up close and personal) espionage. A lot of the most capable people got out over the next two decades. Recruiting became difficult. But after September 11, 2001, the CIA was tossed a huge pile of money and told to staff up and get going and save us all from the Islamic terrorists. The Church Committee restrictions were largely, if not completely, ignored. But long lists of things-you-couldn't-do were still on the books. After a decade of doing whatever it took, the rules are being enforced again.
But now the CIA is back to where it was in 2001, scrambling to rebuild a Cold War era force of spies who knew how to operate in Russia. That capability was not damaged by the Church Committee as much as espionage in other parts of the world (especially Latin America, Africa and Asia) was. Unlike 2001, where experienced agents who had worked in Pakistan during the 1980s (to support operations against Russia in Afghanistan) could be recalled from retirement, most of the “Russia veterans” are largely too old to bring back.
This means the CIA will have to rely on an expedient used after 2001 and that was the use of contractors to run Pakistan intel operations. Things were not done by the book, but results were demanded, especially efforts to find Osama bin Laden. Everyone looked the other way while the deed was done. By 2011 it was official policy to consider Iraq operations over and Afghanistan winding down. At that point Congress was again calling for investigations and “rogue operators” to punish. This sort of thing makes it very difficult to recruit and keep competent spies, even as contractors. It will not be easy to put together a useful espionage operation to deal with Russia.
But it's not just paper bullets intelligence operatives have to worry about these days. The post-9/11 world dramatically altered the way that national intelligence services do business. For one, the craft of espionage and military intelligence has become inherently more dangerous for case officers and agents in an age of terrorism and insurgency than it was during the Cold War.
This is a complete turnaround from the way business was done during the Cold War in the '60s, '70s, and '80s. Many case officers on all sides, whether CIA, Mi6, or KGB, served out their entire 20 or 30 year careers as professional spies without ever having touched a firearm after their initial tradecraft courses. After all, getting into gun battles was not their job. Collecting information was. Furthermore, the case officers themselves, often operating under official diplomatic cover, didn't really have anything to fear if they were caught or their covers blown, except a ruined career and expulsion from whatever country they operated in. The ones in real danger were always the informants, or "assets", that the case officers recruited, who were liable to face execution if they were found out. Simply put, spying really wasn't that dangerous for the case officers.
After the War on Terrorism began, the Cold War rules began to rapidly disappear. For one thing, Afghanistan, Pakistan, and Iraq, along with most places that CIA officers operate today, are actual war zones with nothing "cold" about them. During traditional peacetime case officers don't really have to worry about their own safety, just that of their informants. Once you get involved with terrorists or an actual shooting war starts, all of that changes, and intelligence officers (whether CIA or Army Intelligence) become major high-value targets for terrorist and insurgents. Since 2001, over a dozen (the exact number is classified) CIA officers have been killed in the line of duty. In short, the espionage business has gotten far more dangerous in a very short period of time.
This has necessitated a number of dramatic changes in the way the Americans, British, and other professional intelligence services do business where they are needed most (in war zones). For one, the spooks are getting strapped. Case officers working in places like Pakistan, Afghanistan, and even Egypt routinely carry handguns everywhere they go to defend themselves should the need arise. During the Cold War this was unnecessary and generally considered a stupid liability since being caught with a weapon would probably get you booted out of the country you operated in. Not anymore.
Besides carrying guns, agencies and case officers are paying extra attention to things like counter-surveillance, disguises, and evasive driving. Carrying a sidearm is necessary for a case officer working in a city like Karachi or Kabul, the truth remains that getting into a gunfight is still the last resort and should be avoided at all costs. Case officers know that the most effective way to avoided being a terrorist target is to avoid following the same routines every day, varying routes to and from work/meetings, never sleeping in the same safe house for too long, and generally making one's life as varied and unpredictable as possible. Experienced spies know that if you can't be found, you can't be a target. The best game plan is to be as invisible as possible. Using contractors to run your informant networks is the best cover of all, unless Congress is looking for someone to prosecute.
Russia is different, as the Russians always had the best spies (because of superior recruiting, training, and management). A lot of those spies were cut loose after the Soviet Union dissolved in 1991, and some of them offered to talk (if the price was right). What these guys revealed was chilling for Western intel agencies, a decades long tale of successful old-school espionage operations. The KGB was so good that most of these ops were not even suspected, especially the XX ones. But the new information enabled the U.S. to roll up a number of well-placed Russian agents and moles and provide evidence supporting calls for a return to traditional espionage. Congress was still hostile to that and the September 11, 2001 attacks were one result. The current comedy of errors in Russia is another. There will be more.