May 20, 2017: The United States has long sought to keep secret details of intelligence gathering but over years, or decades, details emerge that confirm suspicions of who was doing what, with what to accomplish specific tasks. In early 2017 it was confirmed (perhaps by accident) that much of the effort to take down ISIL (Islamic State in Iraq and the Levant) since it appeared in 2014 has been the work of U.S. Joint Special Operations Command (JSOC). The 2017 revelations confirmed that JSOC had been responsible for most of the casualties ISIL had suffered so far. JSOC, using a combination of SOCOM (Special Operations Command) operators, similar forces from allies (both NATO and Arab as well as, unofficially, Israel) and a growing number of contractors (usually for SOCOM and other military personnel) were responsible for locating most of the 70,000 ISIL personnel killed so far. JSOC did a lot of the intel work using special equipment and techniques developed and used heavily in Iraq and Afghanistan after 2005. Back then about 90 percent of American special operations forces were in the Middle East, mostly Iraq. But after 2008, when al Qaeda defeated and on the run, SOCOM brought a lot of its personnel back home for some rest and retraining for counter-terror operations elsewhere. Then ISIL showed up in 2014 and now about 60 percent of American special operations forces are in the Middle East and JSOC is planning and coordinating most of the operations. To do this JSOC has call on all American military assets and personnel. That’s where the J, for Joint becomes so important.

In addition to spy satellites and UAVs JSOC was a particularly heavy user of MC-12W intelligence collecting aircraft used by the U.S. (army, air force and JSOC contractors). The MC-12W is basically a militarized version of the commercial Beech King Air. That made it ideal for JSOC because King Airs are widely used by civilian firms worldwide. The U.S. Army began using this Beech aircraft as the RC-12 in the 1970s, and was seeking a replacement when the U.S. Air Force realized that the RC-12 was suitable for use as a Predator substitute (to do surveillance and intel collection). The first air force MC-12Ws entered service over Iraq and Afghanistan by 2009. By 2014 the U.S. Army and SOCOM were known to be taking over RC-12Ws and adding more electronics and new software turned that them into army EMARSS (Enhanced Medium Altitude Reconnaissance and Surveillance System) aircraft. These aircraft can detect, monitor and jam all forms of wireless communications in an area. Since Islamic terrorists tend to lack any sophisticated military-grade communications gear an MC-12W overhead is very bad news. After 2011 most MC-12s were sent to Afghanistan, and found the experience (and success) there similar to that in Iraq. But by 2014 the MC-12s and a lot of UAVs were assigned to JSOC to go after ISIL and that meant returning to Iraq as well as showing up in Africa and anywhere else that ISIL was active.

JSOC was formed in 1980 in the aftermath of a failed mission into Iran to rescue American embassy personnel being held captive there. JSOC was meant to eliminate the coordination problems between the services that were found to be the main reason the Iran rescue mission failed. But until the 1990s JSOC didn’t have much to do and was pretty much a headquarters with no combat troops to command. That changed in the 1990s as the CIA began to suffer from the 1980s move (demanded by Congress and media driven public opinion) to get out of the spying (using people on the ground) business. The CIA found that relying on satellite and aircraft surveillance did not get the job done. It was found that using Delta Force and SEAL commandos for CIA and other intelligence operations worked. This makes sense, because there were still situations overseas, often unanticipated ones, where you really, really needed to get an American in there to look around or make contact with local agents.

By the late 1980s there was another big change, SOCOM was created to manage and control SEALs, Special Forces and other special operations forces. That made it easier for JSOC to easily obtain the use of Delta Force and Seal Team Six operators for jobs like espionage. While many of these operators look like pro football players (kinda makes them stand out, even in civilian clothes), many do not. Turning one of these guys into a secret agent is apparently not difficult. And they already have a license to kill and are very good at handling emergencies and desperate situations.

During the 1990s SOCOM, Delta Force and the SEALs were already prepared to collect intel on the ground themselves simply because they could no longer rely on the CIA to do it for them. That changed after September 11, 2001 but during the 1990s the commandos got experience doing espionage and demonstrated they were very good at it. Despite some new gadgets (UAVs controlled via satellite and information analyzing software) there is still a need to put people on the ground (CIA or SOCOM) to collect information needed before commandos can plan and execute a successful mission. This proved invaluable for winning the war against Islamic terrorists. This is one reason why everywhere Islamic terrorists operate they understand the importance of crippling the local intelligence effort, especially the local officers and bureaucrats in charge of intel, because it is through these men that JSOC delivers information and helps improve local collection.

For a long time no one has said much, officially, about the use of American commandos as spies and intelligence agents. This was in large part because few people outside SOCOM, JSOC and the CIA knew this was going on. Moreover these lads were very effective because they were trained to be flexible, think fast, and operate under any conditions. While they normally train wearing camouflage uniforms and carrying an assault rifle, they have no problem going on a recon mission in a suit, armed with a 9mm pistol, or no weapon at all.

Delta continues to learn from British SAS, who have long made good use of close working relationships with British diplomats and overseas sources. Thus it was no surprise when it became known that after 2001 JSOC operators (Delta and Seal Team Six) began performing secret operations (that stayed secret) involving anti-terrorism activities or nastier forms of diplomacy. Some of these missions did get a lot of publicity, like the capture of Saddam Hussein or the killing of Osama Bin Laden. But JSOC largely stayed in the shadows, or at least tried to.

JSOC also stayed small, growing from about a thousand personnel in 2001 to about 6,000 today (compared to 80,000 for all of SOCOM). The official JSOC personnel are probably even fewer than 6,000 because JSOC can also call on just about any American military personnel they need for individual missions. But to keep JSOC effective it was found that keeping the organization small was essential.

The most successful intelligence gathering technique used by JSOC against ISIL was first developed to detect roadside bombs and the people who made and placed them in Iraq and Afghanistan. This was known to the public as Task Force Odin and by 2006 this effort was using manned (usually MC-12s) and UAV aerial reconnaissance aircraft, along with pattern analysis and data mining, to find IEDs (roadside bombs), and the people who plant them, before the convoys arrive.

To do this, the U.S. Army developed an image analysis system that's basically just another form of pattern analysis. However, it's been a very successful system when it comes to finding newly planted IEDs. The army named this system, Constant Hawk, one of the top ten inventions of 2006. The army names this top ten each year to give some of the more obscure, yet very valuable, developments some well-deserved recognition. If you want a good idea of what the future holds, check lists like this.

What JSOC and Constant Hawk took advantage of was tech that first appeared in the 1930s and 40s. Pattern analysis is one of the fundamental tools Operations Research (OR) practitioners have been using since World War II (when the newly developed field of OR got its first big workout). Pattern analysis is widely used on Wall Street, by engineers, law enforcement, marketing specialists, and now, the military. Constant Hawk uses a special video camera system to observe a locality and find useful patterns of behavior. Some of the Constant Hawk systems are mounted on light (MC-12s, mainly) aircraft, others are mounted on ground structures. Special software compares photos from different times. When changes are noted, they are checked more closely, which has resulted in the early detection of thousands of roadside bombs and terrorist ambushes. This largely eliminated roadside bomb attacks on supply convoys in Iraq, which travelled the same routes all the time. But those routes were also watched by Constant Hawk. No matter what the enemy did, the Hawk noticed. Eventually, the Hawk, and several other efforts, morphed into a campaign in Iraq that led to the death of over 3,000 terrorists caught in the act of setting up roadside bombs, or lying in wait to set them off and attack their victims with gunfire. Hundreds more terrorists were captured, and many thousands of roadside bombs were avoided or destroyed before they could go off.

Despite initial skepticism all this geekery worked, and the troops like tools of this sort mainly because the systems retain photos of areas they have patrolled, and allows them to retrieve photos of a particular place on a particular day. Often, the troops returning from, or going out on a patrol, can use the pattern analysis skills we all have, to spot something suspicious, or potentially so.

A related math tool is predictive analysis. This was widely used in Iraq to determine the identity of bombers, where they were, and where they were most likely to place their bombs next. This enabled the geeks-with-guns (the Army OR specialists) to offer regular "weather reports" about expected IED or terrorist activity. The troops took these reports very seriously, especially those who ran the hundreds of daily convoys that moved people and supplies around Iraq. If your route was predicted to be "hot", you paid extra attention that day, and often spotted IEDs that, as predicted, were there. Usually, the predictions were used to send the combat engineers and EOD (Explosive Ordnance Disposal) teams out to scout and clean the route. It's the feedback from these guys that has brought the geeks their reputation. If the geeks, and their tools (computers, aerial images, and math), say there is something bad out there, they are generally right. For the geeks, it's all pretty obvious. Given enough data, you can predict all sorts of things, or just about anything, really. But to many people, including most reporters, it's all still magic. Task Force Odin is the latest name for an effort that has been going on for over four years, and traces its origins back to World War II, and the invention of Operations Research in the decade before that.

After 2008 JSOC backed further development of these analytic tools and quietly lined up the money to keep the MC-12s flying (often with civilian crews) and making it impossible for ISIL, which was founded by the al Qaeda who survived the 2008 defeat, to find a way to maintain enough secrecy to survive.