2.6.2 Computing information scores for questions
Course subject(s)
Module 2. Calibration and Information score
Intrinsic range
The information score is computed for every question, and by considering all the experts’ assessments for that question.
In computing the information scores (but also in determining the distribution of experts), we need first to provide the support of the experts’ distribution. This is obtained by considering first
L=minimum(all experts 5% quantile, realization)
U=maximum (all experts 95% quantile, realization)
and then the intrinsic range is determined by
[L*,U*]=[L-0.1*(U-L), L+0.1*(U-L)],
for a 10% overshoot rule.
Information score
Once the intrinsic range is computed, the information score is given by
I(e)=0.05*ln(0.05/(q5-L*))+0.45*ln(0.45/(q50-q5)) +0.45*ln(0.45/(q95-q50))+ 0.05*ln(0.05/(U*-q95))+ln(U*-L*)
We again consider the 3 experts who gave their assessments for the 5 questions concerning the Dutch food consumption. The table below shows the assessments:
| Question | Realization | Expert 1
5% 50% 95% |
Expert 2
5% 50% 95% |
Expert 3
5% 50% 95% |
|---|---|---|---|---|
| 1 | 50 | 44 46 49 | 30 40 55 | 38 47 55 |
| 2 | 7 | 9 12 15 | 1 15 20 | 2 8 17 |
| 3 | 108 | 102 106 110 | 60 80 95 | 91 99 106 |
| 4 | 66 | 55 59 64 | 53 70 80 | 58 68 75 |
| 5 | 24 | 28 31 35 | 10 19 30 | 26 35 43 |
Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.