Determining the properties of a newly developed test for comparing receiver operating characteristic (roc) curves

Abstract

Receiver operating characteristic (ROC) curves are graphical plots used for visualizing the performance of binary classifiers. A commonly used summary statistic to describe the ROC curve is its Area Under the Curve (AUC). The AUC’s can be estimated either parametrically or non- parametrically. The parametric approach assumes that the signal present and signal absent groups can be represented by two overlapping Gaussian distributions. A novel asymptotic test for comparing multiple AUC’s of several ROC curves was considered for this study. The objective of this study is to verify the properties of the proposed test. A simulation study was carried out for the case where the AUC’s are independent and to study the behavior of the test for various sample sizes and varying degrees of overlap between the Gaussian distributions. Inferences were made regarding the Type I error and power of the test for the varying parameters. The proposed test performed better with respect to sample sizes above 140 when 3 ROC curves were being compared simultaneously. When the overlap between the Gaussian distributions were less the test statistic performed better with respect to the power of the test.