Born in Moscow, Kazhdan was educated at the Pedagogical Institute of Ufa and the University of Moscow, where he studied with the historian of medieval England, Evgenii Kosminskii. A post-war Soviet initiative to revive Russian-language Byzantine studies led Kazhdan to write a dissertation on the agrarian history of the late Byzantine empire Despite a growing reputation in his field, anti-Semitic prejudice in the Joseph Stalin-era Soviet academy forced Kazhdan to accept a series of positions as a provincial teacher. Following the death of Stalin in 1953, however, Kazhdan's situation improved, and he was hired by a college in Velikie Luki. In 1956 he finally secured a position in the Institute for History of the Soviet Academy of Sciences, where he remained until leaving the Soviet Union in 1978. Kazhdan was an immensely prolific scholar throughout his Soviet career, publishing well over 500 books, articles, and reviews, and his publications contributed to the growing international prestige of Soviet Byzantine studies. His 1954 article, "Vizantiyskie goroda v VII-XI vv.," published in the journal Sovetskaya arkheologiya, argued on the basis of archaeological and numismatic evidence that the seventh century constituted a major rupture in the urban society of Byzantium. This thesis has since been widely accepted and has led to intensive research on discontinuity in Byzantine history and the subsequent rejection of the earlier conception of the medieval Byzantine empire as a frozen relic of late antiquity. Other major studies dating from this first half of Kazhdan's career include Derevnya i gorod v Vizantii IX-X vv., a study of the relationship between city and countryside in the 9th and 10th centuries; Vizantiyskaya kul'tura , a study of Middle Byzantine culture; and Sotsial'ny sostav gospodstvujushchego klassa Vizantii XI-XII vv., an influential prosopographical and statistical study of the structure of the Byzantine ruling class in the eleventh and twelfth centuries. Kazhdan also contributed heavily to the field of Armenian studies, notably writing about the Armenians who formed the elite ruling classes that governed the Byzantine Empire during the MiddleByzantine Era in his Armiane v sostave gospodstvuyushchego klassa Vizantiyskoy imperii v XI-XII vv..
In 1975, Kazhdan's son, the mathematician David Kazhdan, emigrated to the United States, where he accepted a position at Harvard University. This produced an immediate change in Kazhdan's situation in the Soviet Union; his wife, Musja, was fired from her position at a Moscow publishing house and censorship of his work by his superiors in the Soviet academic establishment increased. In October 1978 Alexander and Musja left the Soviet Union, having received a visa for immigration to Israel, coming to the United States three years afterward. In February 1979 they arrived at Dumbarton Oaks, a center for Byzantine studies in Washington, D.C., where Kazhdan held the position of senior research associate until his death. Kazhdan's first major publications in English were collaborative: People and Power in Byzantium, a broad ranging study of Byzantine society, was written with Giles Constable; Studies in Byzantine literature with Simon Franklin; and Change in Byzantine Culture in the Eleventh and Twelfth Centuries with Ann Wharton Epstein. His greatest English-language project was likewise a massive collaborative effort: the three-volume Oxford Dictionary of Byzantium, edited by Kazhdan, was the first reference work of the sort ever to be published, and remains an indispensable point of departure for all areas of Byzantine studies. He wrote approximately 20%, or about 1,000, of the entries in the Dictionary, which are signed with his initials A.K. As Kazhdan became more comfortable with English, his pace of publication once again matched that of his Russian years. His later scholarship is above all marked with a growing concern with Byzantine literature, particularly hagiography. Kazhdan died in Washington, D.C. in 1997. His death cut short his work on a monumental History of Byzantine Literature; however, the first volume of this work, covering the period from 650 to 850, was published in 1999.