Averaging distributions or averaging quantiles

You have seen how averaging weights produce a different set of estimates than when averaging distributions.

The differences were not that big though. However, in practice, there might be situations which lead to big differences in the estimations.

Can you think of an example which will lead to such big differences?

Let’s consider this in detail.

Let’s consider an example where two experts provide their assessments for the probability of dike failure in some specific severe conditions.

Suppose one expert considers the probability to be quite low (so the event of failure to be improbable), whereas the second experts considers the probability to be high. Below are the experts’ quantiles (in percentages and purely fictional!)

E1: 2 (5%), 5 (50%), 10 (95%)
E2: 75 (75%), 85 (50%), 90 (95%)

Suppose that assessments of some calibration questions lead to the performance-based weights: w1=0.9 and w2=0.1.

The two decision makers, PWDM and EWDM lead to the following quantiles (check this!)

PWDM: 2 (5%), 6 (50%), 86 (95%)

EWDM: 2 (5%), 46 (50%), 90 (95%)

Note how, even though the Expert 1 has a very high weight, the distribution of the PWDM is also influenced by the assessments of Expert 2. The contribution of Expert 2 is captured in the uncertainty, that is, in the upper bound of the credible interval (86). Nonetheless, according the PWDM’s solution, values smaller than 6 are more likely than values higher than 6. This is because the 0.45 mass is spread on [2,6] for the second inter-quantile range, whereas mass 0.45 is spread over the interval [6,86] for the third inter-quantile ranges.

Also note that averaging distributions lead to a best estimate of 46. This would say that there is an approximately 50/50 chance that the dike will fail. Along with the uncertainty bounds, we can conclude that the solution of the EWDM is not quite informative. Can you imagine what the real decision makers would be able to do with such assessments?