The McCarthy subcommittee then turned to the overseas library program of the International Information Agency. Cohn toured Europe examining the card catalogs of the State Department libraries looking for works by authors he deemed inappropriate. McCarthy then recited the list of supposedly pro-communist authors before his subcommittee and the press. The State Department bowed to McCarthy and ordered its overseas librarians to remove from their shelves "material by any controversial persons, Communists, fellow travelers, etc." Some libraries actually burned the newly forbidden books. Shortly after this, in one of his carefully oblique public criticisms of McCarthy, President Eisenhower urged Americans: "Don't join the book burners ... Don't be afraid to go in your library and read every book."
Early in 1954, the U.S. Army accused McCarthy and his chief counsel, Roy Cohn, of improperly pressuring the Army to give favorable treatment to G. David Schine, a former aide to McCarthy and a friend of Cohn's, who was then serving in the Army as a private. McCarthy claimed that the accusation was made in bad faith, in retaliation for his questioning of Zwicker the previous year. The Senate Permanent Subcommittee on Investigations, usually chaired by McCarthy himself, was given the task of adjudicating these conflicting charges. Republican Senator Karl Mundt was appointed to chair the committee, and the Army–McCarthy hearings convened on April 22, 1954.
The hearings lasted for 36 days and were broadcast on live television by ABC and DuMont, with an estimated 20 million viewers. After hearing 32 witnesses and two million words of testimony, the committee concluded that McCarthy himself had not exercised any improper influence on Schine's behalf, but that Cohn had engaged in "unduly persistent or aggressive efforts". The committee also concluded that Army Secretary Robert Stevens and Army Counsel John Adams "made efforts to terminate or influence the investigation and hearings at Fort Monmouth", and that Adams "made vigorous and diligent efforts" to block subpoenas for members of the Army Loyalty and Screening Board "by means of personal appeal to certain members of the [McCarthy] committee".
Of far greater importance to McCarthy than the committee's inconclusive final report was the negative effect that the extensive exposure had on his popularity. Many in the audience saw him as bullying, reckless, and dishonest, and the daily newspaper summaries of the hearings were also frequently unfavorable. Late in the hearings, Senator Stuart Symington made an angry and prophetic remark to McCarthy: "The American people have had a look at you for six weeks," he said. "You are not fooling anyone." In Gallup polls of January 1954, 50% of those polled had a positive opinion of McCarthy. In June, that number had fallen to 34%. In the same polls, those with a negative opinion of McCarthy increased from 29% to 45%. He was censured by the Senate in December of 1954. McCarthy's biographers agree that he was a changed man after the censure; declining both physically and emotionally, he became a "pale ghost of his former self" in the words of Fred J. Cook.
This was, then, the man that I rode the trolley with in the Spring of 1957. It was reported that McCarthy suffered from cirrhosis of the liver and was frequently hospitalized for alcoholism. Numerous eyewitnesses, including Senate aide George Reedy and journalist Tom Wicker, have reported finding him alarmingly drunk in the Senate. Journalist Richard Rovere (1959) wrote:
He had always been a heavy drinker, and there were times in those seasons of discontent when he drank more than ever. But he was not always drunk. He went on the wagon (for him this meant beer instead of whiskey) for days and weeks at a time. The difficulty toward the end was that he couldn't hold the stuff. He went to pieces on his second or third drink. And he did not snap back quickly.
For some time opponents of McCarthy had been accumulating evidence concerning his homosexual activities. Several members of his staff, including Roy Cohn and David Schine, were also suspected of having a sexual relationship. Although well-known by political journalists, the first article about it did not appear until Hank Greenspun published an article in the Las Vegas Sun in 25th October, 1952. Greenspun wrote that: "It is common talk among homosexuals in Milwaukee who rendezvous in the White Horse Inn that Senator Joe McCarthy has often engaged in homosexual activities."
McCarthy considered a libel suit against Greenspun but decided against it when he was told by his lawyers that if the case went ahead he would have to take the witness stand and answer questions about his sexuality. In an attempt to stop the rumors circulating, McCarthy married his secretary, Jeannie Kerr. Later the couple adopted a five-week old girl from the New York Foundling Home.