Efficient sequential probability assessment heuristic in decision analysis
Many decision problems involve situations where the possible outcomes are specified but the corresponding probability mass function is only partially known. In such cases, the expected utility of an alternative is not explicitly computable and decisions are made without full information. To address this problem, previous research has tried to establish dominance, by determining if one alternative has a larger expected value or utility than other alternatives for all feasible distributions. In practice, however, dominance is rarely established directly and decision maker needs to make many probability assessments. This work addresses the difficult problem of how to efficiently make probability assessments in decision analysis with complicated uncertainties. After study of the problem, we formulate the problem of achieving dominance with least probability assessments as a dynamic decision problem that shows a huge complexity. A novel heuristic called Sequential Probability Assessment Heuristic (SPAH) is proposed to offer decision analysts a practical way to solve the problem. This method iteratively selects a feasible assessment question for decision analyst to present to the decision maker or expert for assessment in their communication. The heuristic is borrowed from machine learning and, as we show, displays some desirable properties. It performs well when applied to two canonical example decision analysis problems, Eagle Airlines, a decision whether to purchase a plane, and Wildcatter’s valuation, a decision whether to explore an oil well. Our method shows a close performance when compared to the optimal strategy that is solved with the clairvoyance of the true distribution, and a dominating performance over the current standard way of doing probability assessments. The assessment strategy generated by SPAH can also give the decision analyst more insight into the structure of the decision problem they face, as finally we will see from the two examples.