1.4.2 Examples of SEJ studies using Cooke’s Method

Course subject(s) Module 1. When and how to use SEJ?

As mentioned beforehand, there are numerous studies where the Classical Model for Structured Expert Judgment has been applied to aid decision making when data is lacking.

Please review and consider the following four examples of such studies, along with references to the research articles.

Salmonella Bacteria, by NAID, is CC2.0 licensed

In an impressive study conducted together with the World Health Organization (WHO), researchers were able to estimate the global burden of foodborne diseases (FBDs). FBDs include parasites and bacteria, such as Giardia, Toxoplasma gondii, Salmonella and E-coli.

Recently the World Health Organization, Foodborne Disease Burden Epidemiology Reference Group (FERG) estimated that 31 foodborne diseases (FBDs) resulted in over 600 million illnesses and 420,000 deaths worldwide in 2010. Knowing the relative role importance of different foods as exposure routes for key hazards is critical to preventing illness. This study reports the findings of a structured expert elicitation providing globally comparable food source attribution estimates for 11 major FBDs in each of 14 world subregions.

We present the first worldwide estimates of the proportion of specific foodborne diseases attributable to specific food exposure routes. While we find substantial uncertainty around central tendency estimates, we believe these estimates provide the best currently available basis on which to link FBDs and specific foods in many parts of the world, providing guidance for policy actions to control FBDs.

(Hoffmann, Devleesschauwer, Aspinall, Cooke, Corrigan, Havelaar & Speybroeck, 2017)

Article: Hoffmann, S., Devleesschauwer, B., Aspinall, W., Cooke, R., Corrigan, T., Havelaar, A. & Speybroeck, N. (2017). Attribution of global foodborne disease to specific foods: Findings from a World Health Organization structured expert elicitationPLoS One12(9), e0183641. (Open access)

Climate change predictions have been the subject of 3 studies using the Classical Model. The most recent study has appeared in April 2019 in PNAS (open source) and concerns future sea level rise predictions

Future sea level rise (SLR) poses serious threats to the viability of coastal communities, but continues to be challenging to project using deterministic modeling approaches. Nonetheless, adaptation strategies urgently require quantification of future SLR uncertainties, particularly upper-end estimates. Structured expert judgement (SEJ) has proved a valuable approach for similar problems. Our findings, using SEJ, produce probability distributions with long upper tails that are influenced by interdependencies between processes and ice sheets. We find that a global total SLR exceeding 2 m by 2100 lies within the 90% uncertainty bounds for a high emission scenario. This is more than twice the upper value put forward by the Intergovernmental Panel on Climate Change in the Fifth Assessment Report.”
Bamber, J. L., Oppenheimer, M., Kopp, R. E., Aspinall, W. P., & Cooke, R. M. (2019). Ice sheet contributions to future sea-level rise from structured expert judgment. Proceedings of the National Academy of Sciences116(23), 11195-11200.

You can also read about previous study involving climate change

  1. Cooke, R. M. (2013). Uncertainty analysis comes to integrated assessment models for climate change… and conversely.Climatic change,117(3), 467-479.
    2. Bamber, J. L., & Aspinall, W. P. (2013). An expert judgement assessment of future sea level rise from the ice sheets. Nature Climate Change3(4), 424.
    3. Oppenheimer, M., Little, C. M., & Cooke, R. M. (2016). Expert judgement and uncertainty quantification for climate change. Nature climate change6(5), 445.

“The Two Degree Celsius Series” by Tom Hegen is licensed under CC BY-NC-ND 4.0

brand new study reveals the importance of communicating uncertainty about climate change to public.

“The more specific climate scientists are about the uncertainties of global warming, the more the American public trusts their predictions, according to new research by Stanford scholars.”

This nicely connects with our previous focus on uncertainty quantification.

Nuclear safety has actually been the setting in which numerous aspects of the current methodology have been developed in the 90s.

Two studies which are referenced below focused on the quantitative assessments of probability distributions for important parameters in accident consequence models for nuclear power stations, which have been derived and used in the uncertainty analysis of accident consequences.

“Cooling Tower” by Avius Quovis is licensed under CC BY-NC 2.0

1. Goossens, L. H. J., & Harper, F. T. (1998). Joint EC/USNRC expert judgement driven radiological protection uncertainty analysis. Journal of Radiological Protection18(4), 249.
2. Goossens, L. H. J., & Kelly, G. N. (2000). Expert judgement and accident consequence uncertainty analysis (COSYMA)

Volcano activity has been a prolific topic for performing SEJ over the years. Willy Aspinall has used the Classical Model to forecast the course and timing of eruptions in a number of case studies, some of which are listed below.

Volcano erupting on Montserrat.
“File:Monserrat – Soufrière Hills close-up.jpg” by Pat Hawks is licensed under CC BY 3.0

Aspinall, W., & Cooke, R. M. (1998, September). Expert judgement and the Montserrat Volcano eruption. In Proceedings of the 4th international conference on Probabilistic Safety Assessment and Management PSAM4 (Vol. 3, pp. 13-18).
2. Aspinall, W. P. (2006). Structured elicitation of expert judgment for probabilistic hazard and risk assessment in volcanic eruptions. Statistics in volcanology1, 15-30.
Aspinall, W., & Blong, R. (2015). Volcanic risk assessment. In The Encyclopedia of Volcanoes (pp. 1215-1231). Academic Press.
Aspinall, W. P., Woo, G., Voight, B., & Baxter, P. J. (2003). Evidence-based volcanology: application to eruption crises. Journal of Volcanology and Geothermal Research128(1-3), 273-285.

Creative Commons License
Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.
Back to top