Lee Abramson reviewed the elicitation questionnaire results from Tuesday afternoon’s session. Overall, the results were good and consistent with expectations. The group tended to perform better on those questions that asked for the ratio of diseases between men in different age ranges (questions 3 and 4 in exercise). The mid value tended to reasonably close to the actual 2000 census values for these questions.

Additionally, the true value was contained with the 50% interquartile region (75% — 25% percentiles) 10 out of 12 times, or 83%, which is quite good.

The average coverage interval for these questions was 71%. The coverage interval should theoretically be 90%, so that group underestimated the uncertainty. Lee Abramson indicated that typically group uncertainty is about Уг of the true value. In other words, people tend to be more confident in their responses than they should be.

The results were not quite as good when the group was asked to provide absolute disease rates for an age category (Question 2). The true value was inside the 50% interquartile range 4/6 times, or 67%. Also, the coverage interval only captured the true value 61% of the time, 10% less than for the relative questions. This performance demonstrates the supposition relative differences between conditions tend to be more accurate than absolute measures for a given condition. This will be a guiding principal in developing the elicitation framework.