Testing for the difference in sensitivities of binary classifiers using survival analysis

Abstract

Sensitivity is an important measure of the performance of a binary classifier in a disease control program among populations at risk. Where one has to choose between binary classifiers, on basis of their sensitivities, a test grounded on the theory is important for viability of the test results. The commonly used procedure namely the area under the receiver operating characteristic curve does not only lack strong theoretical basis but also trades the sensitivity of a classifier with its specificity. Also, the use of relative sensitivity is limited to comparison at single cut-off point of the classifiers. In this study was we provide a procedure for testing for the difference in sensitivities of two or more binary classifiers over a set of cutoffs without reference to their specificities. By observing the cumulative sensitivities over ordered cut-offs, we defined the survival function of the diseased individuals. Low sensitivity results to high survivability. To test for the difference in sensitivities we tested for the difference of the survival curve using the log-rank test. We subjected our approach to two pancreatic cancer classifiers and the test results show that the two were statistically difference. Further studies should focus on survival curves that close at some point.