The use of the area under the ROC curve in the evaluation of machine learning algorithms
摘要:
In this paper we investigate the use of the area under the receiver operating characteristic (ROC) curve (AUC) as a performance measure for machine learning algorithms. As a case study we evaluate six machine learning algorithms (C4.5, Multiscale Classifier, Perceptron, Multi-layer Perceptron, k-Nearest Neighbours, and a Quadratic Discriminant Function) on six "real world" medical diagnostics data sets. We compare and discuss the use of AUC to the more conventional overall accuracy and find that AUC exhibits a number of desirable properties when compared to overall accuracy: increased sensitivity in Analysis of Variance (ANOVA) tests; a standard error that decreased as both AUC and the number of test samples increased; decision threshold independent; and it is invariant to a priori class probabilities. The paper concludes with the recommendation that AUC be used in preference to overall accuracy for "single number" evaluation of machine learning algorithms. © 1997 Pattern Recognition Society. Published by Elsevier Science Ltd.
展开
关键词:
Accuracy measures Cross-validation Standard error The ROC curve The area under the ROC curve (AUC Wilcoxon statistic
DOI:
10.1016/S0031-3203(96)00142-2
被引量:
年份:
1997
相似文献
参考文献
引证文献
辅助模式
引用
文献可以批量引用啦~
欢迎点我试用!