Elicitation

Book excerpt: This book is about elicitation: the facilitation of the quantitative expression of subjective judgement about matters of fact, interacting with subject experts, or about matters of value, interacting with decision makers or stakeholders. It offers an integrated presentation of procedures and processes that allow analysts and experts to think clearly about numbers, particularly the inputs for decision support systems and models. This presentation encompasses research originating in the communities of structured probability elicitation/calibration and multi-criteria decision analysis, often unaware of each other’s developments. Chapters 2 through 9 focus on processes to elicit uncertainty from experts, including the Classical Method for aggregating judgements from multiple experts concerning probability distributions; the issue of validation in the Classical Method; the Sheffield elicitation framework; the IDEA protocol; approaches following the Bayesian perspective; the main elements of structured expert processes for dependence elicitation; and how mathematical methods can incorporate correlations between experts. Chapters 10 through 14 focus on processes to elicit preferences from stakeholders or decision makers, including two chapters on problems under uncertainty (utility functions), and three chapters that address elicitation of preferences independently of, or in absence of, any uncertainty elicitation (value functions and ELECTRE). Two chapters then focus on cross-cutting issues for elicitation of uncertainties and elicitation of preferences: biases and selection of experts. Finally, the last group of chapters illustrates how some of the presented approaches are applied in practice, including a food security case in the UK; expert elicitation in health care decision making; an expert judgement based method to elicit nuclear threat risks in US ports; risk assessment in a pulp and paper manufacturer in the Nordic countries; and elicitation of preferences for crop planning in a Greek region.