Highlights New
web-based
elicitation tool
As part of the MATCH project, we have developed a web-based tool for eliciting probability distributions. The tool implements various elicitation methods that are described within the SHELF package. An important feature of this tool is that users can log in from different sites, and all see and interact with the same graphics. Handy if you can't get your experts together in the same room!
Elicitation
software and papers
I have written some elicitation functions in R that are available within the SHELF package (release 2.0). These functions implement the bisection/quartile, trial roulette, fixed interval, and tertile methods for elicitation, using interactive graphics (courtesy of the excellent rpanel package!) I have also written a tutorial paper on univariate elicitation, and co-written a tutorial paper on multivariate elicitation, which will be both be appearing in the edited volume “Re-Thinking Risk Measurement, Management and Reporting – Measurement Uncertainty, Bayesian Analysis and Expert Elicitation”.
Go to the SHELF homepage to download SHELF 2.0
Download univariate elicitation tutorial paper
Download multivariate elicitation tutorial paper (supporting R code for this chapter is available here)
Managing
Uncertainty in Complex Models (MUCM) and
the MUCM
Toolkit
The MUCM project is developing a technology that is capable of addressing all sources of uncertainty in model predictions and to quantify their implications efficiently, even in the most complex models. It has the potential to revolutionise scientific debate by resolving the contradictions in competing models. It will also have a radical effect on everyday modelling and model usage by making the uncertainties in model outputs transparent to modellers and end users alike. Online resources are available at the MUCM toolkit.
I am a senior lecturer in the School of Mathematics and Statistics at the University of Sheffield. I obtained my BSc (1996) in Mathematics and Statistics from the University of Nottingham, and my PhD (2000) in Statistics from the University of Sheffield. I have worked as a postdoctoral research associate in both the Department of Computer Science and Department of Probability and Statistics, University of Sheffield, before starting a lectureship in Probability and Statistics in 2002.
My research interests are in Bayesian inference, and in particular...
Complex computer models are in widespread use to simulate complex real-world processes. There is great concern about the accuracy of such models, and how to quantify the various uncertainties in their use. Innovative Bayesian methods are providing powerful tools to answer these and other important questions facing users of such models.
Eliciting
prior distributions from experts
Elicitation
is the process of extracting expert knowledge about some unknown
quantity of interest, or the probability of some future event, which
can then be used to supplement any numerical data that we may have.
If the expert in question does not have a statistical background, as
is often the case, translating their beliefs into a statistical form
suitable for use in our analyses can be a challenging task.
The
SHeffield ELicitation Framework (SHELF)
SHELF is a package of documents, templates and software to carry out elicitation of probability distributions for uncertain quantities from a group of experts, developed by Tony O’Hagan and myself, and is available for download free of charge.
As part of the MATCH project, we have developed a web-based tool for eliciting probability distributions. The tool implements various elicitation methods that are described within the SHELF package. An important feature of this tool is that users can log in from different sites, and all see and interact with the same graphics. Handy if you can't get your experts together in the same room!
The
main concern of the field of health economics is to examine the
cost-effectiveness of medical technologies. The Department of
Probability and Statistics, in collaboration with Sheffield's School
of Health and Related Research (ScHARR),
has established the Centre
for Bayesian Statistics in Health Economics.
For
more information please visit the department's
Ren,
S. and Oakley, J. E. Assurance
calculations for planning clinical trials with time-to-event
outcomes.
Submitted to Statistics
in Medicine. Download
supporting R code.
Strong,
M. and Oakley, J. E. Is
my model good enough? Deriving the expected value of model
improvement via specifying model discrepancy.
Submitted to Journal
of Uncertainty Quantification.
Oakley,
J. E., Daneshkhah, A. and O’Hagan, A. Nonparametric
elicitation using the roulette method.
Submitted to Bayesian
Analysis.
Strong,
M. and Oakley, J. E. An
efficient method for computing partial expected value of perfect
information for correlated inputs.
To appear in Medical
Decision Making.
Fricker, T. E., Oakley, J. E.
and Urban, N. M. Multivariate
Gaussian process emulators with nonseparable covariance structures.
To appear in Technometrics
Becker,
W., Oakley, J. E., Surace, C., Gili, P., Rowson, J., & Worden, K.
(2012). Bayesian sensitivity analysis of a nonlinear finite element
model. To appear in Mechanical
Systems and Signal Processing.
Strong,
M., Oakley
J. E. and Chilcott, J. (2012). Managing
structural uncertainty in health economic decision models: a
discrepancy approach.
Journal of the Royal
Statistical Society, Series C,
61(1), 25-45.
Wilkinson,
R. D., Vrettas, M., Cornford, D. and Oakley, J. E. (2011).
Quantifying simulator discrepancy in discrete-time dynamical
simulators. Journal
of Agricultural, Biological, and Environmental Statistics,16(4),
554-570.
Fricker,
T. E.,
Oakley
J. E., Sims, N. D. and Worden, K. and Chilcott, J. (2011).
Probabilistic uncertainty analysis of an FRF of a structure using a
Gaussian process emulator. Mechanical
Systems and Signal Processing, 25(8),
2962-2975.
Oakley,
J. E. (2011). Modelling with deterministic computer models. In
Simplicity,
Complexity and Modelling,
M. Christie, A. Cliffe, P. Dawid and S. Senn (eds.). Chichester:
Wiley.
Becker, W.,
Rowson, J., Oakley J. E., Yoxall, A., Manson, G. and Worden K.
(2011). Bayesian sensitivity analysis of a model of the aortic valve.
Journal
of Biomechanics 44(8),
1499-1506.
Oakley, J.
E. and Clough, H. E. (2010) Sensitivity analysis in microbial risk
assessment: vero-cytotoxigenic E.coli
O157 in
farm-pasteurised milk. Handbook
of Applied Bayesian Analysis,
O’Hagan, A. and West, M. (eds). Oxford University Press.
Conti, S., Gosling, J. P.,
Oakley, J. E. and O'Hagan, A. (2009). Gaussian process emulation of
dynamic computer codes. Biometrika
96,
663-676.
Oakley, J.
E. (2009). Decision-theoretic sensitivity analysis for complex
computer models. Technometrics
51,
121-129.
Oakley, J.
(2004). Estimating percentiles of computer code outputs. Journal
of the Royal Statistical Society, Series C,
53, 83-93.
Oakley, J.
and O'Hagan, A. (2004). Probabilistic sensitivity analysis of complex
models: a Bayesian approach. Journal
of the Royal Statistical Society Series B,
66, 751-769. Download
example data.
Oakley, J. and O'Hagan, A.
(2002). Bayesian inference for the uncertainty distribution of
computer model outputs. Biometrika,
89, 769-784.
Oakley, J. (2002). Eliciting
Gaussian process priors for complex computer codes. The
Statistician, 51,
81-97.
O'Hagan, A.,
Kennedy. M. C. and Oakley, J. E. (1999). Uncertainty analysis and
other inference tools for complex computer codes (with discussion).
In Bayesian Statistics 6, J. M. Bernardo et al (eds.). Oxford
University Press, 503-524.
Oakley, J. E. (2010). Eliciting
univariate probability distributions,
in Rethinking
Risk Measurement and
Reporting: Volume I ,
edited by Böcker, K., Risk Books, London.
Daneshkhah
A. and Oakley, J.E. (2010). Eliciting
multivariate probability distributions.
(Supporting R code for this chapter is available here)
in Rethinking
Risk
Measurement and Reporting: Volume I ,
edited by Böcker, K., Risk Books, London.
Gosling, J.P., Oakley, J.E. and
O'Hagan, A. (2007). Nonparametric elicitation for heavy-tailed prior
distributions. Bayesian
Analysis, 2,
693-718.
Oakley, J.
and O'Hagan, A. (2007). Uncertainty in prior elicitations: a
non-parametric approach. Biometrika
94,
427-441.
O' Hagan,
A., Buck, C. E., Daneshkhah, A., Eiser, J. E., Garthwaite, P. H.,
Jenkinson, D. J., Oakley, J. E. and Rakow, T. (2006). Uncertain
Judgements: Eliciting Expert Probabilities.
Chichester: Wiley.
O'Hagan, A.
and Oakley, J. E. (2004). Probability is perfect, but we can’t
elicit it perfectly. Reliability
Engineering and System Safety,
85, 239-248.
Strong,
M. and Oakley, J. E. An
efficient method for computing partial expected value of perfect
information for correlated inputs.
To appear in Medical
Decision Making.
Strong,
M., Oakley
J. E. and Chilcott, J. (2012). Managing structural uncertainty in
health economic decision models: a discrepancy approach.
Journal of the Royal Statistical Society, Series C,
61(1), 25-45
Strong,
M. and Oakley
J. E. (2011). Bayesian inference for comorbid disease risks using
marginal disease risks and correlation information from a separate
source. Medical
Decision Making 31(4),
571-581.
Available
online here.
Oakley,
J.E., Brennan, A., Tappenden, P. and Chilcott, J.B. (2010). Sample
sizes for Monte Carlo partial EVPI calculations.
Journal
of Health Economics 29(3),
468-77. Download
example R code.
Stevenson,
M. D., Oakley, J. E., Lloyd Jones, M. Brennan, A.,
Compston, J. E. , McCloskey E. V. and Selby P. L. (2009). The
Cost-Effectiveness of an RCT to Establish Whether 5 or 10 Years of
Bisphosphonate Treatment Is the Better Duration for Women With a
Prior Fracture. Medical
Decision Making 29(6),
678-689.
Stevenson, M. D., Oakley, J.
E., Chick, S. E. and Chalkidou, K. (2009). The cost-effectiveness of
surgical instrument management policies to reduce the risk of vCJD
transmission to humans. Journal
of the Operational Research Society 60,
506-518.
Coyle D. and Oakley J. (2008)
Estimating the expected value of partial perfect information: a
review of methods. The
Eur. Journal of Health Economics 9,
251-259.
Karnon J.,
McIntosh A., Coster J., Bath P., Hutchinson A., Oakley J., Thomas N.,
Pratt P., Freeman-Parry L., Karsh B. T., Gandhi T. and Tappenden T.
(2008). Modelling the expected net benefits of interventions to
reduce the burden of medication errors. Journal
of Health Services Research and Policy 13,
85-91.
Stevenson, M. D., Lloyd Jones,
M., De Nigris E., Brewer, N., Davis, S. and Oakley, J. (2005). A
systematic review and economic evaluation of alendronate, etidronate,
risedronate, raloxifene and teriparatide for the prevention and
treatment of postmenopausal osteoporosis. Health
Technology Assessment,
Vol.9: No. 22.
Stevenson,
M. D., Brazier, J. E., Calvert, N.W., Lloyd-Jones M., Oakley, J. and
Kanis, J.A. (2005). Description of an individual patient methodology
for calculating the cost-effectiveness of treatments for osteoporosis
in women. Journal
of the Operational Research Society,
56, 214-221.
Stevenson M.D., Oakley, J. and
Chilcott, J.B. (2004). Gaussian process modelling in conjunction with
individual patient simulation modelling: A case study describing the
calculation of cost-effectiveness ratios for the treatment of
osteoporosis. Medical
Decision Making,
24(1), 89-100.
Tappenden,
P., Chilcott, J. B., Eggington, S., Oakley, J. and McCabe, C. (2004).
Methods for expected value of information analysis in complex health
economic models: developments on the health economics of
beta-inteferon and glatiramer acetate for multiple sclerosis. Health
Technology Assessment,
Vol. 8: No. 27.
Nixon, R.M.,
O'Hagan, A., Oakley, J. E., Madan, J., Stevens, J.W. Bansback, N. and
Brennan, A. (2009) The Rheumatoid Arthritis Drug Development Model: A
case study in Bayesian clinical trial simulation. Pharmaceutical
Statistics 8(4),
371-389
Karnon, J.,
McIntosh, A., Bath, P., Dean, J., Hutchinson, A., Oakley, J., Thomas,
N., Pratt, P., Freeman-Parry, L., Karsh, B., Gandhi, T. and
Tappenden, P. (2007). Medication errors: a prospective hazard and
improvement analysis. Safety
Science 45,
523-539.
Jeremy
Oakley
The University Of Sheffield
School of Mathematics and
Statistics
The Hicks Building
Hounsfield Road
Sheffield
S3 7RH
United Kingdom
Phone:
+44-(0)114-2223853
Fax: +44-(0)114-2223809