Herschmann was Jewish, and was born in Vienna, Austria. He was affiliated with the 1.W.A.S.C. in Vienna, and the Wiener AC in Vienna.
Olympic swimming career
Herschmann first competed at the initial modern Olympic Games, the 1896 Summer Olympics in Athens, Greece, at the age of 19 in the men's 100 metres freestyle swimming event. On 30 March, he and the other swimmers were taken by boat into the Bay of Piraeus to compete in the open sea. The competitors swam from a starting line between two buoys, through a course marked by a number of floating hollow pumpkins, to a red flag finish line at the shore. Herschmann placed second and won a silver medal, with a time of 1:22.8, 0.6 seconds and half a metre behind the winner, Alfréd Hajós, as the other swimmers trailed far behind. AinsworthSports.com ranked Herschmann as tied for the second-best swimmer of the 1890s, behind Alfréd Hajós. In 1904, he wrote Wiener Sport, which was published by H. Seemann.
In the 1906 Summer Olympics, Herschmann competed in Athens in individual sabre, but did not medal. He returned to Olympic competition at the 1912 Summer Olympics in Stockholm, Sweden, competing as a member of Austria's sabre fencing team at the age of 35, 16 years after he first won a medal. On 15 July he won a silver medal in the team competition. In so doing, he became one of only a few athletes to win Olympic medals in more than one sport. Other Jewish fencers who participated in the 1912 Olympics included Hungarian gold-medal winning sabre fencers Dr. Jenő Fuchs, Dr. Dezső Földes, Lajos Werkner, and Dr. Oszkár Gerde, and Austrian silver-medal winning sabre fencerAlbert Bogen.
Athletic administrative posts
At the time he won his fencing medal, Herschmann was serving as President of the Austrian Olympic Committee, a position that he held from 1912 to 1914. He is the only person to win an Olympic medal while serving as president of a National Olympic Committee. Herschmann was one of Europe's top authorities in sports. In November 1913, he traveled to various cities in the United States, including Boston, New York, Cleveland, Philadelphia, and Chicago, to study US sports organizations and recruit trainers to work with Austrian athletes training for the Olympics. That month, when he was visiting the U.S. as the Austrian athletic envoy, the Boston Athletic Association gave him a banquet, and in December 1913 the Board of Governors of the New York Athletic Club held a banquet honoring him. He lauded the United States system for the quality of physical and mental training provided. He noted in contrast to the European system, high-quality training was provided to all athletes, not only those who lacked natural talent. Herschmann served as President of the Austrian Swimming Federation from 1914 to 1932.
Concentration camp and death
Herschmann was in private practice as a lawyer in the 1940s. He was persecuted during the era of the Nazis because he was Jewish. Herschmann was deported from Vienna on 14 January 1942 to Izbica concentration camp in German-occupied Poland. Herschmann died on 14 June 1942 in Sobibór extermination camp in Sobibór, Lubelskie, Poland, most likely by being gassed to death.