Correct evaluation of the accuracy of medical tests may be important for helping clinicians avoid errors.
The accuracy of medical tests is important for minimizing errors and their possible sequelae. “Accuracy of a diagnostic test” is a term that is frequently used loosely to describe the evaluation of a medical test versus a gold standard —for example, the detection (in the general population) or diagnosis (in the patient population) of cardiovascular disease using a stress test versus catheterization as a gold standard. Similarly, such an evaluation is performed using prostate-specific antigen (PSA) for detecting or diagnosing prostate cancer versus a biopsy as a gold standard. Some textbooks and publications use a specific measure of “overall accuracy” of a test that is the ratio of correct diagnoses to all diagnoses (correct and incorrect) in a 2×2 table. It is thus the proportion of the correct test results.1
Alberg recommended cautious use of the overall accuracy measure, because it does not take into account the true prevalence of the disease and therefore is misleading. A similar cautious approach was also advocated by others. Our manuscript addresses this problem.
We suggest that there should be a clear distinction between the overall accuracy measures of a test aimed at the detection of a disease in a screening setting in a population for public health purposes in the general population and the overall accuracy measures of a test aimed at determining a diagnosis of individuals in a clinical setting in the patient population. The overall detection accuracy measure is obtained in a specific study that samples persons with known diagnoses, and may be useful for public health screening tests. It is different from the overall diagnostic accuracy that is calculated in the clinical setting, sampling individuals with a positive or a negative test result. We thus suggest using two distinct overall accuracy measures: the overall detection accuracy, which is applicable in the screening and public health settings, and the overall diagnostic accuracy, which is applicable in the clinical setting and is dependent on the prevalence of the disease (that is, the proportion of persons with the disease). This new measure may be important for helping clinicians avoid errors.
The assessment of a diagnostic test is frequently based on a study in a selected population, sampled according to the disease status, and is determined according to the gold standard. The study is used for calculating the sensitivity and specificity. the sampling is according to disease status (sick versus not sick), and thus only the totals in the columns are meaningful. The data in this table are defined by the test performance among already diagnosed persons (with or without a disease). These data are important for detecting a disease in a population and are useful in a public health setting and for decision making. For example, one may evaluate how many of the sick and healthy persons may be detected by a test for a disease among passengers in a transportation vehicle, and thus assess the resources needed in various public health and disease control settings. Such data are useful for choosing the appropriate (that is, the most efficient and least costly) test in a given population with a known and constant disease prevalence.
It is important to use accurate medical tests and thus avoid errors and unnecessary suffering and expenses. As already mentioned by Alberg et al. and others, overall accuracy measures that do not take into account the true prevalence of the disease may be misleading.
Our manuscript addresses this problem and suggests a clear distinction between the overall detection accuracy (which does not take the prevalence into account) and the overall diagnostic accuracy, which does. We suggest that the overall detection accuracy is calculable in a screening setting in populations; it may be useful for public health purposes, but it is meaningless in the clinical setting. The overall diagnostic accuracy, which is calculable in the patient population based on the true prevalence, is more informative to the patient and the physician.
It may clarify the use and interpretation of test results and could avoid confusion that may result from ignoring the disease prevalence in measuring the test overall accuracy. Correct evaluation of the accuracy of medical tests may be important for helping clinicians avoid errors.