One reason the war on terror is so difficult to deal with is that it feeds on extremist beliefs within the Moslem community. That is a big problem because there are over 1.2 billion Moslems worldwide and they can be found in just about every country. Worse, modern communications and relatively inexpensive international travel has made it much easier, than in any past time, for terrorists to move about and keep in touch with each other. For this reasons the lessons of Iraq, Afghanistan, and other campaigns in the war on terror are not as useful as they could be.
At the same time American involvement in local wars of this sort of effort is nothing new. Since the 19th century the United States has been involved in over sixty of them. The American military has, since the 1970s, sought to study this past experience because it was recognized that capturing lessons from these past conflicts was an important and valuable undertaking. The military, especially the army and marines, knew from experience that past experience is indeed very useful, as they recently discovered in Iraq and Afghanistan. At the same time the troops discovered that new elements, like 24 hour video news and worldwide news networks and the Internet changed the public perceptions of these wars and willingness to let the troops take care of it. As a result more of these small wars are dragging on and on with no end in sight.
While all this awareness of the reality of small wars was useful for the military and intelligence agencies it proved very difficult to get the national leadership and the public to understand the nature of these wars. Actually, from the very beginning there was a reluctance among American officials to reveal the masses of data collected and how it was analyzed. Partly this was to prevent the enemy from realizing how much information on terrorist operations it possessed. But another reason was the fact that such a large mass of data could be interpreted many different ways, some of them unfavorable to the United States. Thus there was no "body count" or any other type of measure released by the Department of Defense after Vietnam. Internally, there were various metrics (measurements) presented to senior military and political leadership. The big problem was the use of aggregation (combining a lot of data together that should not have been combined). That was a problem that slowly became obvious after 2001.
It's now recognized that the wars in Iraq and Afghanistan (and elsewhere, like Yemen, Somalia, North Africa, and so on) were all somewhat different and that context for each of them was crucial if you were going to analyze them. For example, al Qaeda is more of an idea than a centralized organization. Thus the al Qaeda found in each country, or part of a country, usually has different means and motivations. The war in Iraq was actually several separate wars going on at the same time and occasionally interacting with other "wars" nearby. Same thing happened in Afghanistan and places like Somalia. Measuring progress is more accurate if you show the unique trends in all the different wars. Some of them ended early, some escalated, and some are still in progress, while others evolve into new kinds of conflicts. In other words, the military should use contextual assessment in reporting what is going on with guerilla conflict (or "irregular warfare" in general).
What the Department of Defense is doing here is fixing a problem it created. This happened during the Vietnam War, which the U.S. military establishment didn't really want any part of. The Pentagon in the 1960s was fixated on the conventional military threat the Soviet Union presented in Europe and elsewhere. But a new generation of technocrats had taken over in the Department of Defense and many of their new ideas were handed over to bureaucrats who didn't understand what they were doing. Generals who pointed out problems in these new methods tended to retire ahead of schedule.
Before long it became conventional wisdom that the U.S. was incapable of handling irregular warfare. This was odd, as the United States had an enviable track record when it comes to defeating guerillas and irregular forces in general. Even Vietnam, which conventional wisdom counts as a defeat, wasn't. The conventional wisdom, as is often the case, is wrong. By the time the last U.S. combat units pulled out of South Vietnam in 1972, the local guerilla movement, the Viet Cong, was destroyed. North Vietnam came south three years later with a conventional invasion, sending tank and infantry divisions charging across the border and conquering their neighbor the old fashioned way.
When the United States first got involved with Vietnam in the late 1950s, there was good reason to believe American assistance would lead to the defeat of the communist guerilla movement in South Vietnam. That was because the communists had not been doing so well with their guerilla wars. In the previous two decades there had been twelve communist insurgencies, and 75 percent of them had been defeated. These included the Greek Civil War (1944-1949), Spanish Republican Insurgency (1944-1952), Iranian Communist Uprising (1945-1946), Philippine Huk War (1946-1954), Madagascan Nationalist Revolt (1947-1949), Korean Partisan War (1948-1953), Sarawak/Sabah "Confrontation" (1960-1966), Malayan Emergency (1948-1960), and the Kenyan Mau-Mau Rebellion (1952-1955). The communists won in the Cuban Revolution (1956-1958), the First Indochina War (1945-1954), and the Chinese Civil War (1945-1949). The communists went on to lose the guerilla phase of the Second Indochina War (1959-1970).
Guerillas make great copy for journalists. You know, the little guy, fighting against impossible odds. What we tend to forget (and the record is quite clear and easily available), is that these insurgent movements almost always get stamped out. That does not make good copy and the dismal details of those defeats rarely make it into the mass media or the popular consciousness. Few of the current “gray zone” conflicts involve communists but they are all ideological to one degree or another and many feature diehard communists or radical leftists among the fighters and terrorists.
The main problem with COIN (Counterinsurgency Warfare) is that the American armed forces take it for granted. U.S. troops have been defeating guerilla movements for centuries. Through most of American history COIN has been the most frequent form of warfare American troops were involved in. But COIN has always been viewed as a minor, secondary military role. It never got any respect. The generals preferred to prepare for a major war with a proper army, not playing cops and robbers with a bunch of poorly organized losers.
Even the U.S. Marine Corps, after half a century of COIN operations, were glad to put that behind them in the late 1930s. All that remained of that experience was a classic book, "The Small Wars Manual," written by some marine officers on the eve of World War II. That book, which is still in print, contained timeless wisdom and techniques on how to deal with COIN operations and "small wars" in general. Every COIN book since simply repeats the basic wisdom laid down in "The Small Wars Manual."
The basic truth is that COIN tactics and techniques have not changed for thousands of years. What has also not changed is the professional soldiers disdain for COIN operations. This sort of thing has never been considered "real soldiering." But the U.S. Army and Marines have finally come to accept that COIN is a major job, something that U.S. troops have always been good at and something that you have to pay attention to.
So when you see more news stories about new COIN manuals or American ignorance of irregular warfare, keep in mind the history of that kind of warfare and how long, and successfully, Americans have been dealing with it.