3.8.2 Expert selection

Course subject(s) Module 3. Performance-based weights and the Decision Maker

So removing an expert can change the remaining experts’ information score and therefore their performance-based weights.

The next question that you might ask yourself is whether there will consequently be an effect on the performance of the DMs?

Let’s see!

After Expert A have been removed, the information scores of Expert B and Expert C have changed. The changes can be found in the table below.

Calibration score Information score
Expert B 0.608 0.220
Expert C 0.327 0.971

The updated performance-based weights are 0.3 (for Expert B) and 0.7 (for Expert C).

These weights are somewhat different from the initial performance-based weights 6.56E-005 (Expert A), 0.43 (Expert B) and 0.57 (Expert C).

Let’s see if the updated weights will lead to differences in the two decision makers.

The updated performance-based weights lead to the following assessments for the Performance-based Decision Maker (PWDM) and the Equal Weight Decision Maker (EWDM).

Question Realization Expert B

5% 50% 95%

Expert C

5% 50% 95%

PWDM

5% 50% 95%

EWDM

5% 50% 95%

1 24 20 23 34 20  25  28 20  24.68  32.31 20  24.38  33.15
2 11 8 10 11 10  13  15 8.33  11.71  14.92 8.15  10.75  14.82
3 68 60 70 90 65  67  70 61.24  67.34  86.28 60.61  67.69  88.24
4 47 20 35 45 42  44  46 23.03  43.3  45.98 21.39  42.5  45.96
5 31 30 35 40 32  33  35 30.61  33.29  39.1 30.31  33.57  39.58

As a comparison, here are the initial assessments obtained for the two decision makers.

Question Realization Expert A

5% 50% 95%

Expert B

5% 50% 95%

Expert C

5% 50% 95%

PWDM

5% 50% 95%

EWDM

5% 50% 95%

1 24 2 5 7 20 23 34 20 25 28 19.97 24.48 33.11 2.55 21.77 32.39
2 11 12 13 15 8 10 11 10 13 15 8.19 10.91 14.86 8.31 12.23 14.91
3 68 87 89 92 60 70 90 65 67 70 60.75 67.56 87.94 61.43 71.08 91.61
4 47 48 50 52 20 35 45 42 44 46 21.02 42.8 45.97 2.55 34.28 45.78
5 31 2 5 8 30 35 40 32 33 35 30.29 33.47 39.82 30.66 35.34 51.63

Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.