3.8.2 Expert selection

Course subject(s) Module 3. Performance-based weights and the Decision Maker

So removing an expert can change the remaining experts’ information score and therefore their performance-based weights.

The next question that you might ask yourself is whether there will consequently be an effect on the performance of the DMs?

Let’s see!

After Expert A have been removed, the information scores of Expert B and Expert C have changed. The changes can be found in the table below.

	Calibration score	Information score
Expert B	0.608	0.220
Expert C	0.327	0.971

The updated performance-based weights are 0.3 (for Expert B) and 0.7 (for Expert C).

These weights are somewhat different from the initial performance-based weights 6.56E-005 (Expert A), 0.43 (Expert B) and 0.57 (Expert C).

Let’s see if the updated weights will lead to differences in the two decision makers.

The updated performance-based weights lead to the following assessments for the Performance-based Decision Maker (PWDM) and the Equal Weight Decision Maker (EWDM).

Question	Realization	Expert B 5% 50% 95%	Expert C 5% 50% 95%	PWDM 5% 50% 95%	EWDM 5% 50% 95%
1	24	20 23 34	20 25 28	20 24.68 32.31	20 24.38 33.15
2	11	8 10 11	10 13 15	8.33 11.71 14.92	8.15 10.75 14.82
3	68	60 70 90	65 67 70	61.24 67.34 86.28	60.61 67.69 88.24
4	47	20 35 45	42 44 46	23.03 43.3 45.98	21.39 42.5 45.96
5	31	30 35 40	32 33 35	30.61 33.29 39.1	30.31 33.57 39.58

As a comparison, here are the initial assessments obtained for the two decision makers.

Question	Realization	Expert A 5% 50% 95%	Expert B 5% 50% 95%	Expert C 5% 50% 95%	PWDM 5% 50% 95%	EWDM 5% 50% 95%
1	24	2 5 7	20 23 34	20 25 28	19.97 24.48 33.11	2.55 21.77 32.39
2	11	12 13 15	8 10 11	10 13 15	8.19 10.91 14.86	8.31 12.23 14.91
3	68	87 89 92	60 70 90	65 67 70	60.75 67.56 87.94	61.43 71.08 91.61
4	47	48 50 52	20 35 45	42 44 46	21.02 42.8 45.97	2.55 34.28 45.78
5	31	2 5 8	30 35 40	32 33 35	30.29 33.47 39.82	30.66 35.34 51.63

Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.