2.6.3 Computing the information score for experts
Course subject(s)
Module 2. Calibration and Information score
We saw how the information score is computed for each question.
For the example of the Dutch eating habits, we obtain the following results.
Question | Expert 1 | Expert 2 | Expert 3 |
---|---|---|---|
1 | 1.33 | 0.04 | 0.30 |
2 | 0.87 | 0.12 | 0.16 |
3 | 1.56 | 0.28 | 0.98 |
4 | 0.89 | 0.05 | 0.34 |
5 | 1.28 | 0.39 | 0.53 |
Note that the calculations for the intrinsic range of Question 2 are the following:
– [L,U]=[1,20]
– the overshoot is k*(U-L)=0.1*19=1.9
Extending the interval with the overshoot would lead to a negative lower bound. Nonetheless, the question is about the percentage of Dutch adults eating fast food less than once a month. So a negative % doesn’t make too much sense.
For this reason, the intrinsic range [L*,U*]=[0,21.9] has been chosen.
In order to obtain an overall score that describes how informative expert’s assessments are, we average the information scores for all questions.
So, for the 5 questions, the information score of each expert is
Expert 1 | Expert 2 | Expert 3 | |
---|---|---|---|
Information score | 1.19 | 0.18 | 0.46 |
Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.