3.8.2 Expert selection
Course subject(s)
Module 3. Performance-based weights and the Decision Maker
So removing an expert can change the remaining experts’ information score and therefore their performance-based weights.
The next question that you might ask yourself is whether there will consequently be an effect on the performance of the DMs?
Let’s see!
After Expert A have been removed, the information scores of Expert B and Expert C have changed. The changes can be found in the table below.
| Calibration score | Information score | |
|---|---|---|
| Expert B | 0.608 | 0.220 |
| Expert C | 0.327 | 0.971 |
The updated performance-based weights are 0.3 (for Expert B) and 0.7 (for Expert C).
These weights are somewhat different from the initial performance-based weights 6.56E-005 (Expert A), 0.43 (Expert B) and 0.57 (Expert C).
Let’s see if the updated weights will lead to differences in the two decision makers.
The updated performance-based weights lead to the following assessments for the Performance-based Decision Maker (PWDM) and the Equal Weight Decision Maker (EWDM).
| Question | Realization | Expert B
5% 50% 95% |
Expert C
5% 50% 95% |
PWDM
5% 50% 95% |
EWDM
5% 50% 95% |
|---|---|---|---|---|---|
| 1 | 24 | 20 23 34 | 20 25 28 | 20 24.68 32.31 | 20 24.38 33.15 |
| 2 | 11 | 8 10 11 | 10 13 15 | 8.33 11.71 14.92 | 8.15 10.75 14.82 |
| 3 | 68 | 60 70 90 | 65 67 70 | 61.24 67.34 86.28 | 60.61 67.69 88.24 |
| 4 | 47 | 20 35 45 | 42 44 46 | 23.03 43.3 45.98 | 21.39 42.5 45.96 |
| 5 | 31 | 30 35 40 | 32 33 35 | 30.61 33.29 39.1 | 30.31 33.57 39.58 |
As a comparison, here are the initial assessments obtained for the two decision makers.
| Question | Realization | Expert A
5% 50% 95% |
Expert B
5% 50% 95% |
Expert C
5% 50% 95% |
PWDM
5% 50% 95% |
EWDM
5% 50% 95% |
|---|---|---|---|---|---|---|
| 1 | 24 | 2 5 7 | 20 23 34 | 20 25 28 | 19.97 24.48 33.11 | 2.55 21.77 32.39 |
| 2 | 11 | 12 13 15 | 8 10 11 | 10 13 15 | 8.19 10.91 14.86 | 8.31 12.23 14.91 |
| 3 | 68 | 87 89 92 | 60 70 90 | 65 67 70 | 60.75 67.56 87.94 | 61.43 71.08 91.61 |
| 4 | 47 | 48 50 52 | 20 35 45 | 42 44 46 | 21.02 42.8 45.97 | 2.55 34.28 45.78 |
| 5 | 31 | 2 5 8 | 30 35 40 | 32 33 35 | 30.29 33.47 39.82 | 30.66 35.34 51.63 |
Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.