In statistics, Barnard's test is an exact test used in the analysis of contingency tables. It examines the association of two categorical variables and is a more powerful alternative than Fisher's exact test for 2×2 contingency tables. While first published in 1945 by George Alfred Barnard, the test did not gain popularity due to the computational difficulty of calculating the p-value and Fisher's disapproval. Nowadays, for small/moderate sample sizes, computers can often implement Barnard's test in a few seconds.
Purpose and scope
Barnard's test is used to test the independence of rows and columns in a contingency table. The test assumes each response is independent. Under independence, there are three types of study designs that yield a 2×2 table. To distinguish the different types of designs, suppose a researcher is interested in testing whether a treatment quickly heals an infection.
One possible study design would be to sample 100 infected subjects, randomly give them the treatment or the placebo, and see if the infection is still present after a set time. This type of design is common in cross-sectional studies.
Another possible study design would be to give 50 infected subjects the treatment, 50 infected subjects the placebo, and see if the infection is still present after a set time. This type of design is common in case-control studies.
The final possible study design would be to give 50 infected subjects the treatment, 50 infected subjects the placebo, and stop the experiment once a set number of subjects has healed from the infection. This type of design is uncommon, but has the same structure as the ‘lady tasting tea’ study that led to R. A. Fisher creating Fisher's Exact test.
The probability of a 2×2 table under the first study design is given by the multinomial distribution; the second study design is given by the product of two independent binomial distributions; the third design is given by the hypergeometric distribution. The difference between Barnard's exact test and Fisher's exact test is how they handle the nuisance parameter of the common success probability when calculating the p-value. Fisher's test avoids estimating the nuisance parameter by conditioning on the margins, an approximately ancillary statistic. Barnard's test considers all possible values of the nuisance parameter and chooses the value that maximizes the p-value. Both tests have sizes less than or equal to the type Ierror rate. However, Barnard's test can be more powerful than Fisher's test because it considers more ‘as or more extreme’ tables by not conditioning on both margins. In fact, one variant of Barnard's test, called Boschloo's test, is uniformly more powerful than Fisher's exact test. A more detailed description of Barnard's test is given by Mehta and Senchaudhuri. Barnard's test has been used alongside Fisher's Exact test in project management research
Criticisms
Under pressure from Fisher, Barnard retracted his test in a published paper, however many researchers prefer using Barnard's exact test over Fisher's exact test for analyzing 2×2 contingency tables. The only exception is when the true sampling distribution of the table is hypergeometric. Barnard's test can be applied to larger tables, but the computation time increases and the power advantage quickly decreases. It remains unclear which test statistic is preferred when implementing Barnard's test; however, most test statistics yield uniformly more powerful tests than Fisher's exact test.
