Direct Weights
The Busara Clinical Decision Making Framework (BCDMF) deliberative phase is designed to be used when decision makers are not ready to make a decision after examining decision-related data and tradeoffs intuitively. Use of the deliberative phase should be considered anytime decisions present difficult tradeoffs and/or when making high stakes decisions, particularly those that cannot be reversed, such as having surgery.
The BCDMF is based on multi-criteria decision analysis (MCDA). MCDA is designed to help people make better choices when decisions involve tradeoffs between competing decision objectives, a characteristic of many medical decisions. There are a number of well developed MCDA methods. They all use the same basic decision model but differ in the method used to identify preferred alternatives. A nice feature is that the methods can progressively build on each other, so it is possible to increase the complexity of an analysis without needing to start over. [1]
The methods included in the Busara Clinical Decision Making Framework are listed in the following table. They can be applied to both assessing the relative priorities of the decision criteria and how well the options meet the criteria.
All of these methods work by creating quantitative scales that reflect decision makers judgments about how well the options meet the criteria and the priorities of the criteria relative to the goal. These scales help characterize the judgments being made in the decision making process more exactly than possible using qualitative terms or intuitive feelings. They therefore provide a new and enhanced way for decision makers to communicate with each other about their preferences and priorities. They also enable decision makers to explore how changing their initial preference and priority judgments affects the overall assessments of the options under consideration.
In last week’s Musing (April 28, 2023), I described the rank order weighting method. The beauty of the rank order weights is their simplicity. Once the rankings are established the work is done. However, how well they work depends on how accurately the rank order weights match the judgments of the decision maker(s). For this reason, the BCDMF tool contains a direct weighting module that allows decision makers to adjust the weights that have been automatically assigned by the ranking process. (This module can also be used directly – there is no need to do the ranking first.)
To review, suppose a doctor and a patient are choosing among 3 possible treatment options using the following decision model.
They collect data on how well each option meets the three criteria and rank order them for best to worst in each category. The results of the example ranking and ordinal rank weights are shown below.
They also rank order and weight the three decision criteria in terms of how important they are in meeting the goal of picking the best initial treatment option:
Now let’s assume that our example patient does not agree with these rank-assigned weights. She and her physician therefore decide to use direct weights to adjust them to more closely match her preferences. There are several ways to do this. A common method is to rate the items on a 1-10 scale and then normalize the results by dividing each rating by the sum of all ratings. An example rating process for the options relative to the Response Rate criterion is shown in the following table:
The same procedure is then used to assign priority scores to the decision criteria in terms of how important they are in achieving the goal of the decision:
The analysis is completed using the same method as with ordinal ranking scores. After the options have all been compared relative to the criteria and the criteria compared relative to the goal, overall scores are calculated for each option by multiplying the option criteria weights times the criteria priorities and summing the results, a procedure similar to calculating a weighted average. To make it easier to review and discuss the scores, the scores are multiplied by 100 to remove the decimal places. (See details in the April 28, 2023 Musing.)
If you would like to explore the direct weighting procedure further, I’ve made a Google Sheets file that will do the direct weighting for the example problem. It can be assessed using this link, I hope. If you have problems accessing it, please send me a comment and I will try to fix it.
Musings
Like the ordinal rank weighting methods, the direct weighting method is easy to use and can be programmed into any spreadsheet, so can be implemented quickly and easily. The direct weighting method shares the advantages of the ordinal ranking method but, in addition, can more accurately reflect a decision maker’s decision preferences and priorities than is possible using ordinal rank weights.
In some cases the additional information provided by this analysis will provide enough information to help decision makers reach a decision. If not, the BCDMF provides two additional modules that take a different approach to analyzing a decision that can provide additional insight into a complicated decision making scenario. I will describe these in upcoming Musings.
References
1. Dolan JG. Multi-Criteria Clinical Decision Support: A Primer on the Use of Multiple-Criteria Decision-Making Methods to Promote Evidence-Based, Patient-Centered Healthcare. The Patient: Patient-Centered Outcomes Research. 2010 Dec;3(4):229–48.
mdm787 