3.7.1 The performance of the Decision Maker
You have now learned how to combine experts’ assessments and to obtain the solution of this aggregation in the same format as the experts’ assessments. That is
experts’ quantiles –> experts’ distribution –> combined experts’ distribution (DM’s distribution) –> DM’s quantiles
You used this to find answers (estimates along with uncertainty quantification) for the questions of interest.
But this procedure can be applied not only to the questions of interest, but also to the calibration questions!!
If so, the DM’s assessments (quantiles) could be subjected to the same objective evaluation as any expert’s assessments. And we could then compute the calibration score and the information score of the DM!!
This is made more explicit in the next video!
Evaluating the performance of a DM: calibration and information score
Decision Making Under Uncertainty: Introduction to Structured Expert Judgment by TU Delft OpenCourseWare is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Based on a work at https://online-learning.tudelft.nl/courses/decision-making-under-uncertainty-introduction-to-structured-expert-judgment//.