For thousands of years, torture was a common practice. But a lot of that torture was done not just to obtain information, but more often to terrorize a population or, as fans of violent sports are reluctant to admit, there's entertainment value. The 20th century brought the scientific method to bear on the use of torture. If the objective was mainly to obtain information, be it military secrets or a confession from someone innocent of a crime, then many new forms of torture were developed, perfected and widely used. But one constant was the need for an expert interrogator. This has long been recognized. The infamous medieval Inquisition used a lot more quiet questioning than it did gruesome forms of physical torture. The 20th century torturers also perfected the art of psychological torture. The best criminal investigators have a large bag of interrogation techniques which often obtain the desired information without any physical contact with the suspect. The Soviet Union, over it's seven decades of existence, literally wrote the book on non-physical torture. Part of this was due to the nature of the communist nations. They were police states and were constantly on the lookout for disloyalty. But the Soviet Union had another major advantage; they were able to create thousands of expert, career interrogators. These were men, and a few women, who spent decades perfecting their skills. The Soviet Unions had several college level institutions that amassed and passed on vast amounts of knowledge and technique in the use of physical and psychological torture.
Despite the considerable skills of the communist interrogators, they were not always able to get the information, or confessions, they wanted. World War II saw an enormous amount of torture. The Nazis, who openly admired the superior interrogation skills of the Soviets, were more prone to use poorly trained investigators who went to physical torture quickly. The results were often dismal. Thousands of Russian and Allied victims took their secrets to their (usually unmarked) grave. The Soviets proved that, if you have the time (weeks or months) and skilled interrogators, you can break just about anyone who can be broken. As for the others, a bullet in the back in the head was the usual result. The Soviet interrogators were not good losers.
One of the Soviet techniques that got more attention than success was the use of drugs to loosen tongues. The classic "truth serum" is sodium pentothal, which is basically an anesthetic. Much earlier, booze or drugs were used, with some success, to get people to loosen up and talk. Female spies have long used erotic behavior and sex to get men to spill secrets. Using anesthetics has the advantage of being administered in more controlled doses. During the first few decades of the Cold War, there was something of a "drug race" between the United States and the Soviet Union to develop more effective drugs for use as interrogation tools. LSD, heroin and many exotic chemicals were all used in the search for the perfect truth serum. There were no breakthroughs, but many people in the interrogation business still keep an eye on new developments in anesthesia. And there are new things worth watching. Many new anesthetics don't knock you out as much as they put you in an altered state where you don't notice, or simply don't remember, the pain. Doctors often warn patients that they may want to use another form of anesthetic, because these new ones (using combinations of tranquilizers and sedatives in an IV drip) have a tendency to cause people to say things they'd rather keep secret. No one has admitted to using these new anesthetics for interrogation, probably considering success in this area as a valuable military secret. Or perhaps there is simply worry that such use will simply be outlawed for interrogation.
But when you have to get critical information in a hurry, like where a terrorist group has hidden a bomb (perhaps a small nuclear one) or biological weapon, most people will not quibble about the use of torture. In this case it is literally a matter of life or death. For this reason, most intelligence agencies stay current on torture techniques. Just in case.
Does Torture Work? It depends. It depends on who is asking the questions and who is being asked. An expert interrogator can get information out of almost anyone. But there are many people who resist torture unto death. While some people respond more to torture than to psychological pressure, the widespread use of torture is generally an indication of a lack of experienced interrogators. This is the situation facing the United States now. With over a thousand, mainly Arab, al Qaeda suspects in custody, America doesn't have enough experienced, Arab speaking interrogators to get much out of these suspects. Some of these hard core terrorist suspects may talk under torture, but true believers tend to resist physical pressure well.