Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation
摘要：
The areas beneath the relative (or receiver) operating characteristics (ROC) and relative operating levels (ROL) curves can be used as summary measures of forecast quality, but statistical significance tests for these areas are conducted infrequently in the atmospheric sciences. A development of signal-detection theory, the ROC curve has been widely applied in the medical and psychology fields where significance tests and relationships to other common statistical methods have been established and described. This valuable literature appears to be largely unknown to the atmospheric sciences where applications of ROC and related techniques are becoming more common. This paper presents a survey of that literature with a focus on the interpretation of the ROC area in the field of forecast verification. We extend these foundations to demonstrate that similar principles can be applied to the interpretation and significance testing of the ROL area. It is shown that the ROC area is equivalent to the Mann-Whitney U -statistic testing the significance of forecast event probabilities for cases where events actually occurred with those where events did not occur. A similar derivation shows that the ROL area is equivalent to the Mann-Whitney U -statistic testing the magnitude of events with respect to whether or not an event has been forecast. Because the Mann-Whitney U -statistic follows a known probability distribution, under certain assumptions it can be used to define the statistical significance of ROC and ROL areas and for comparing the areas of competing forecasts. For large samples the significance of either measure can be accurately assessed using a normal-distribution approximation. Copyright 2002 Royal Meteorological Society
展开
关键词：
Forecast verification Mann–Whitney U‐test Probabilistic forecasts Signal‐detection theory Student's t‐test
DOI：
10.1256/003590002320603584
被引量：
年份：
2010
通过文献互助平台发起求助，成功后即可免费获取论文全文。
相似文献
参考文献
引证文献
辅助模式
引用
文献可以批量引用啦~
欢迎点我试用！
