Zakharov participated in the First World War with the rank of second lieutenant, having completed training at a school for ensigns in 1916. In October 1917, he was elected a regimental commander. During the Russian Civil War, from August 1919 he became a company commander in the Red Army, and was in combat on the Eastern Front. Zakharov continued his training throughout the 1920s, including the officer training courses known as Vystrel. He also took a teaching assignment at the Frunze Military Academy from 1933, and underwent further training at the Military Academy of the General Staff in 1939. From 1939–1941 Zakharov served as Chief of Staff of the Ural Military District.
World War II service
During the first days of World War II, Zakharov was serving as Chief of Staff of the 22nd Army, which saw action against the German invasion, Operation Barbarossa, in the environs of Vitebsk, Velikie Luki and Nevel. From August 1941, Zakharov was Chief of Staff and, during October- November, the commander of the Bryansk Front, which covered the Oryol - Tula and L'vov - Kursk axes. From December 1941, Zakharov was deputy Commander of the Western Front, which participated in the counterattack on the approaches to Moscow during the Battle of Moscow. From April 1942, Zakharov was Chief of Staff of the North Caucasian Strategic Direction; from May, the commander of the North Caucasian Front, and; from August, the chief of staff of the Stalingrad Front. From October 1942, Zakharov was the deputy commander of the Stalingrad Front, where he successfully exercised leadership of the armies on the Front's left wing during the Soviet counterattack. During January 1943, Zakharov was appointed Deputy Commander of the Southern Front, while from February he was commander of the 51st Army. From July 1943, Zakharov commanded the 2nd Guards Army, which performed outstandingly during the penetration of the German defences on the Mius and Molochnaya rivers. Under his leadership, the 2nd Guards Army broke through powerful fortified positions on the Perekop Isthmus during April 1944 and participated in the liberation of Sevastopol. During 1944, Zakharov was given the responsibility of commanding the 2nd Belorussian Front during the major strategic offensive, Operation Bagration. The Front's performance during this operation was criticised as inadequate, and from November Zakharov was effectively demoted to command of the 4th Guards Army, which was employed in forcing crossings of the Danube and participated in the envelopment of German forces in Budapest during the Battle of Budapest. In April 1945, Zakharov was appointed Deputy Commander of the 4th Ukrainian Front.
Cold War service
After the war, Zakharov commanded the troops of the Southern-Ural and Eastern Siberian military districts, and was the commander of Vystrel courses. From September 1954, he was the Deputy Department Chief of combat training of ground forces.