University of Surrey

Test tubes in the lab Research in the ATI Dance Research

Classifier calibration using splined empirical probabilities in clinical risk prediction.

Gaudoin, R, Montana, G, Jones, S, Aylin, P and Bottle, A (2014) Classifier calibration using splined empirical probabilities in clinical risk prediction. Health Care Management Science.

[img] Text (licence)
Restricted to Repository staff only
Available under License : See the attached licence file.

Download (33kB)


The aims of supervised machine learning (ML) applications fall into three broad categories: classification, ranking, and calibration/probability estimation. Many ML methods and evaluation techniques relate to the first two. Nevertheless, there are many applications where having an accurate probability estimate is of great importance. Deriving accurate probabilities from the output of a ML method is therefore an active area of research, resulting in several methods to turn a ranking into class probability estimates. In this manuscript we present a method, splined empirical probabilities, based on the receiver operating characteristic (ROC) to complement existing algorithms such as isotonic regression. Unlike most other methods it works with a cumulative quantity, the ROC curve, and as such can be tagged onto an ROC analysis with minor effort. On a diverse set of measures of the quality of probability estimates (Hosmer-Lemeshow, Kullback-Leibler divergence, differences in the cumulative distribution function) using simulated and real health care data, our approach compares favourably with the standard calibration method, the pool adjacent violators algorithm used to perform isotonic regression.

Item Type: Article
Authors :
Gaudoin, R
Montana, G
Jones, S
Aylin, P
Bottle, A
Date : 1 February 2014
DOI : 10.1007/s10729-014-9267-1
Uncontrolled Keywords : Calibration, Probability estimation, Logistic regression, Empirical probabilities, HES data
Related URLs :
Depositing User : Symplectic Elements
Date Deposited : 28 Mar 2017 15:29
Last Modified : 31 Oct 2017 17:10

Actions (login required)

View Item View Item


Downloads per month over past year

Information about this web site

© The University of Surrey, Guildford, Surrey, GU2 7XH, United Kingdom.
+44 (0)1483 300800