Generalization Bounds for the Area Under the ROC Curve
摘要：
We study generalization properties of the area under the ROC curve (AUC), a quantity that has been advocated as an evaluation criterion for the bipartite ranking problem. The AUC is a different term than the error rate used for evaluation in classification problems; consequently, existing generalization bounds for the classification error rate cannot be used to draw conclusions about the AUC. In this paper, we define the expected accuracy of a ranking function (analogous to the expected error rate of a classification function), and derive distribution-free probabilistic bounds on the deviation of the empirical AUC of a ranking function (observed on a finite data sequence) from its expected accuracy. We derive both a large deviation bound, which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on a test sequence, and a uniform convergence bound, which serves to bound the expected accuracy of a learned ranking function in terms of its empirical AUC on a training sequence. Our uniform convergence bound is expressed in terms of a new set of combinatorial parameters that we term the bipartite rank-shatter coefficients; these play the same role in our result as do the standard VC-dimension related shatter coefficients (also known as the growth function) in uniform convergence results for the classification error rate. A comparison of our result with a recent uniform convergence result derived by Freund et al. (2003) for a quantity closely related to the AUC shows that the bound provided by our result can be considerably tighter.
展开
DOI：
10.1007/s10883-005-4175-9
被引量：
年份：
2005
EBSCO
Semantic Scholar
(全网免费下载)
Semantic Scholar
Semantic Scholar
ACM
查看更多
IDEALS
IDEALS
(全网免费下载)
ResearchGate
EBSCO
Citeseer
(全网免费下载)
Citeseer
jmlr.csail.mit.edu
(全网免费下载)
valis.cs.uiuc.edu
(全网免费下载)
sarielhp.org
(全网免费下载)
valis.cs.uiuc.edu
jmlr.org
(全网免费下载)
jmlr.org
jmlr.org
(全网免费下载)
drona.csa.iisc.ernet.in
(全网免费下载)
l2r.cs.uiuc.edu
(全网免费下载)
research.microsoft.com
(全网免费下载)
cogcomp.cs.illinois.edu
(全网免费下载)
cogcomp.org
(全网免费下载)
machinelearning.wustl.edu
core.ac.uk
AMS
microsoft.com
(全网免费下载)
citeseer.uark.edu:8080
core.ac.uk
(全网免费下载)
131.107.65.14
(全网免费下载)
msr-waypoint.com
(全网免费下载)
msr-waypoint.com
discovery.ucl.ac.uk
学术范
citeseer.ist.psu.edu
(全网免费下载)
msr-waypoint.net
131.107.65.14
machinelearning.wustl.edu
(全网免费下载)
mendeley.com
research.microsoft.com
research.microsoft.com
(全网免费下载)
osti.gov
(全网免费下载)
hdl.handle.net
jmlr.csail.mit.edu
sarielhp.org
herbrich.me
(全网免费下载)
zentralblatt-math.org
学术范
(全网免费下载)
Semantic Scholar
(全网免费下载)
IDEALS
(全网免费下载)
Citeseer
(全网免费下载)
osti.gov
(全网免费下载)
jmlr.org
(全网免费下载)
valis.cs.uiuc.edu
(全网免费下载)
drona.csa.iisc.ernet.in
(全网免费下载)
131.107.65.14
(全网免费下载)
cogcomp.org
(全网免费下载)
sarielhp.org
(全网免费下载)
herbrich.me
(全网免费下载)
core.ac.uk
(全网免费下载)
machinelearning.wustl.edu
(全网免费下载)
research.microsoft.com
(全网免费下载)
citeseer.ist.psu.edu
(全网免费下载)
l2r.cs.uiuc.edu
(全网免费下载)
cogcomp.cs.illinois.edu
(全网免费下载)
research.microsoft.com
(全网免费下载)
msr-waypoint.com
(全网免费下载)
jmlr.org
(全网免费下载)
microsoft.com
(全网免费下载)
jmlr.csail.mit.edu
(全网免费下载)
学术范
(全网免费下载)
通过文献互助平台发起求助，成功后即可免费获取论文全文。
相似文献
参考文献
引证文献
辅助模式
引用
文献可以批量引用啦~
欢迎点我试用！
