Interactive guideline decision models & dashboards

In last week’s post, I proposed that guideline creation panels start including decision models and dashboards in their guideline summaries and recommendations. I included an example decision model and dashboard to illustrate how this process could work. The dashboard I included has a couple of shortcomings. It is not interactive and does not show how uncertainties can be included in a dashboard display. The purpose of today’s post is to demonstrate the basic elements of an interactive decision dashboard.

Today’s technology makes it easy to build and disseminate complex, interactive dashboards and other information visualizations of all types. One of the best resources to learn about these capabilities is Tableau Public. Users of this site can explore information visualizations about many topics and learn how to create and post their own. Use of the Tableau Public site is free but all visualizations are open to public access. Licenses for the Tableau software can be purchased for proprietary use.

I used the Tableau Public site to create a simple interactive dashboard regarding the following decision:

Imagine you have a newly diagnosed chronic disease that is causing symptoms severe enough to limit your daily activities. Fortunately,  several treatment options are available. Information about how well they work, their risk of side effects, and monthly out-of-pocket cost is summarized in the table below:

The goal of a decision dashboard is to help you compare the three drugs and choose the one that you would pick to treat your symptoms. A dashboard using the information in the table that compares the 3 drugs is shown below. Note, if you want to choose which drugs to display, use the checkboxes on the upper right.

A screenshot of the dashboard is shown below. Click this link to access the working version.

The initial display compares how the three drugs compare over three different dimensions: disease control or effectiveness, risk of side effects, and out of pocket cost. The range of possible values are graphically illustrated in the different colored bars.

Some may find the default display sufficient to choose a preferred drug. For example, someone may not be able to afford more than $100 per month, which would eliminate Drug C and probably Drug B as well. They can quickly see, however, that Drug A is the safest drug and should be as nearly if not as effective as the other two, making it a good choice.

Others may choose to make a decision by first eliminating a less desirable drug and then reviewing how the remaining two match up. For example, someone might choose to eliminate Drug C due to its high cost and high risk of side effects compared to the other two drugs. They then uncheck the box next to Drug C on the interactive dashboard and concentrate on comparing the two remaining drugs across the three decision criteria as illustrated below:

Musings

Clinical guidelines summarize and evaluate existing data to recommend how it should be applied in clinical practice. Most guidelines as currently written summarize the research findings and list practice recommendations in great detail, sometimes across the span of two or three separate articles. This format does a good job of documenting the data supporting the recommendations but is not designed to make either the data or the recommendations readily usable to support decision making in clinical practice. A simple solution to this problem is to create a decision-ready summary of the guideline data and recommendations in the form of a decision model and interactive decision dashboard.

In addition to making the guideline information more readily usable, a recommendation-based decision model could also be used to collect the perspectives and judgments of diverse clinical decision makers. The model could be disseminated as a small, interactive file with questions regarding decision related trade offs and judgments, and a mechanism for modifying the basic model supplied by the guideline panel. Models adapted through clinical use could be saved and anonymously aggregated to provide important information that could be used to inform future iterations of the practice guidelines.

Knowing is not enough, we must apply

“I have been impressed with the urgency of doing. Knowing is not enough; we must apply. Being willing is not enough; we must do.” ~ Leonardo da Vinci

---

Atrial fibrillation (AF) is the most common arrhythmia in the general population. Anticoagulation to prevent embolic events is a key part of the management of patients with atrial fibrillation. For many years warfarin, a vitamin K antagonist, was the only anticoagulant option. The introduction of a group of direct acting anticoagulants has made the choice of treatment more complicated. Compared with warfarin the newer agents do not require frequent monitoring and are somewhat more effective and safer, but more expensive.

In the February 2023 issue of Medical Decision Making, Kathryn Martinez and colleagues published a brief report describing their analysis of how well physicians engaged patients with atrial fibrillation in shared decision making regarding the choice of anticoagulant. [1] Recorded conversations were evaluated for 7 key elements of shared decision making using a list first developed by Braddock and colleagues in 1999. [2] The authors found that physicians frequently omitted elements of the shared decision making process and, in particular, rarely assessed patient preferences. They concluded:

Multiple professional societies support informed decision making for anticoagulation in patients with AF. Data from these real-world encounters of physicians and patients making these decisions suggest informed decision making is largely not taking place. Use of decision aids to support anticoagulation decisions may facilitate more complete informed decision making.

Musings

This paper is well done and calls appropriate attention to an important, complex decision faced by patients with atrial fibrillation. However I wish it had done more. Many prior studies, including the seminal paper by Braddock that was the source of the study methodology, have found similar results. After more than two decades, I think it is firmly established that physicians do not engage patients in shared decision making as often as they should. The outstanding questions are what can be done about it and who should take the lead in fixing the problem.

Medical education institutions and organizations producing clinical guidelines are the two most obvious places where progress could be made. Medical curricula could be modified to include explicit training in decision making tailored to provide students a firm background in the practice of good decision making, but results will take time. The quicker approach is to adapt practice guidelines to include decision models populated with decision-focused summaries of the research data that was reviewed that support the guideline recommendations.

To illustrate, let’s consider what it would take to create a decision model alongside a clinical guideline. Imagine a guideline panel is meeting to issue guidelines for treatment of a fictitious condition called Hendassa Disease. After reviewing the literature, the panel has decided to endorse four drugs for clinical use: Drugs A,B,C, and D. In addition to a written summary of the data and their recommendation, they create a decision model for an illustrative patient called Anna. The design of a decision model is outlined in the first figure below, the illustrative patient model created by the guideline panel in the second.

The next step for guideline developers would be to summarize information about the four treatment options in a manner that facilitates comparisons between their respective pros and cons. Ideally this would consist of a summary table and a corresponding decision dashboard. Examples of both formats are shown below. Note the dashboard has some example patient data input for this illustration. In practice, the guideline panel would leave this field empty, to be assessed by individual patients, as shown in the table.

This decision model and data summary would then be distributed along with the practice guidelines to provide clinicians with a ready-made platform for informing patients about the decision at hand and inquiring about their decision related preferences and values.

Creating a decision model and decision-oriented data summary like this could potentially increase the utility of guideline research and recommendations while requiring little additional effort by guideline panels. It seems to me someone should give it a try.

References:

1. Martinez, Kathryn A., Debra T. Linfield, Victoria Shaker, and Michael B. Rothberg. 2023. “Informed Decision Making for Anticoagulation Therapy for Atrial Fibrillation.” Medical Decision Making 43(2): 263–69.

2. Braddock III, C. H., Edwards, K. A., Hasenberg, N. M., Laidley, T. L., & Levinson, W. (1999). Informed decision making in outpatient practice: time to get back to basics. Jama, 282(24), 2313-2320.

Managing uncertainty

An essential part of medical decision making

The high degree of consistency in our results, across topics, magnitudes of uncertainty, and communication formats suggest that people “can handle the truth.” [1]

---

Uncertainty affects practically all important medical decisions. Therefore, effective decision aiding methods need to help people manage the uncertainties inherent in decisions they need to make.

There are two types of uncertainty that affect decision making. The first, called aleatory, is an uncertainty that we have no way of knowing in advance, such as the weather next year on Halloween. The second, called epistemic, refers to uncertainties due to the limitations of current knowledge that could be clarified or eliminated through further study.

In the October 28, 2022 Musing, I discussed a paper titled “Current best practice for presenting probabilities in patient decision aids: Fundamental principles” by Carissa Bonner and colleagues. [2] In it, they advised decision creators to “be cautious” about communicating epistemic uncertainties regarding the data included in the aid:

Be cautious about communicating second-order, epistemic uncertainty (e.g., using probability ranges), given that this uncertainty may be psychologically aversive and difficult to understand, and that optimal methods of communication remain to be determined” (Bonner et al., 2021, p. 824)

In contrast to concerns about the potential adverse effects of communicating uncertainties, there are good arguments supporting the need for communicating scientific information accurately and transparently, particularly when the communication is designed to support good decision making.

In 2020, Anne Marthe van der Bles and colleagues published an open access paper in the Proceedings of the National Academy of Sciences describing a series of experiments designed to assess the effects of communicating epistemic uncertainty, quantitatively and using verbal expressions, on people’s cognitions, emotions, and trust. [1] They found little evidence to support withholding information about evidence uncertainty. This is how they summarize their findings:

Overall, we found little evidence to suggest that communicating numerical uncertainty about measurable facts and numbers backfires or elicits psychological reactance. Across five high powered studies and an internal metaanalysis, we show that people do recognize and perceive uncertainty when communicated around point estimates, both verbally and numerically (except when only words such as “estimated” or “about” are used to imply uncertainty). In addition, uncertainty did not seem to influence their affective reaction … and although the provision of uncertainty in general did slightly decrease people’s trust in and perceived reliability of the numbers, this effect emerged for explicit verbal uncertainty in particular.

Musings

Although only one paper, the five studies included in the report by van der Bles and colleagues strongly suggests that gains in insight and transparency are likely to far outweigh the adverse effects of including quantitative information about the uncertainty of data included in decision aids. In addition to the outcomes measured in the study, including uncertainty information will also help people avoid the “flaw of averages”. This concept, developed by Sam Savage, simply states that decisions based on an average value frequently fail because the underlying variation in the quantity being measured is not taken into account. [3]

The key task for decision aid creators, and all those tasked with helping others make good decisions, is to help people effectively manage the uncertainties inherent in the data being considered. One way is to simply present the data graphically. A useful glossary of methods for displaying data distributions is on the Tableau Public website. Additional well-established methods for helping decision makers understand and manage future uncertainties are Monte Carlo simulations and scenario analysis. Many financial institutions use Monte Carlo simulations to project retirement income and savings. Sam Savage and colleagues have constructed an ingenious method of embedding Monte Carlo simulations in individual cells of spreadsheets that could be easily integrated into a medical decision aid that I think merits further investigation. [4]

References

1. van der Bles AM, van der Linden S, Freeman ALJ, Spiegelhalter DJ. The effects of communicating uncertainty on public trust in facts and numbers. Proceedings of the National Academy of Sciences. 2020 Apr 7;117(14):7672–83.

2. Bonner, Carissa et al. 2021. “Current Best Practice for Presenting Probabilities in Patient Decision Aids: Fundamental Principles.” Medical Decision Making 41(7): 821–33.

3. Savage S. The Flaw of Averages. Harvard Business Review [Internet]. 2002 Nov 1 [cited 2023 Mar 15]; Available from: https://hbr.org/2002/11/the-flaw-of-averages

4. Probability Management [Internet]. Probability Management. 2023 [cited 2023 Mar 15]. Available from: https://www.probabilitymanagement.org

Interactive decision dashboards

Moving medical decision making into the 21st Century.

Information visualization is “the use of computer-supported, interactive, visual representations of abstract data to amplify cognition”. [1]

---

In last week’s Musing, I discussed the advantages of using information visualization as a way to quickly and easily enable busy clinical decision makers to engage in deliberative as well as intuitive decision making.

To be successful, clinical information visualizations have to be prepared in advance and easily accessible in the consultation room. With the advent of electronic medical records and ubiquity of smartphones and computer tablets, there is now a convenient, familiar delivery system available, if we choose to use it. The use of electronic tools also easily enables an additional important cognitive advantage of information visualization: the ability to interact with the data in a non-linear, self-directed manner.

For most clinical problems, several considerations will be of interest, such as the chances of a successful outcome, risks of side effects, and costs. A visual display showing a group of considerations and their respective data is called a dashboard.

Dashboards are collections of data visualizations, presented in a single-page view that imparts at-a-glance information on which users can act quickly. [2]

To illustrate, imagine a patient and her doctor are deciding which of four drugs is the best initial treatment for a new problem. Information about the treatment choices is summarized in the table below.

Instead of using this table, they choose to use an interactive dashboard displaying the same information running on a tablet computer. Note that the drug regimen data was first transformed to a numerical preference score running from 5 (most preferred) to 1 (least preferred) using input provided by the patient:

Working with the dashboard, the clinical decision makers first notice that Drug C is a dominated option, never the best on any criterion and often the worst. Therefore they eliminate it from consideration by simply checking a box on the dashboard display and concentrate on comparing the remaining three choices using the revised dashboard shown below:

At this point they note that, in terms of Effectiveness and Risk of Side Effects, the differences among the three options are small. The biggest differences are the costs, that range from $5 to $40 per month.

Drug D, the most expensive option, is only slightly easier to use than Drug B, which costs half as much. Drug D is also the option most likely to have a side effect and not the best in terms of effectiveness. They therefore decide to eliminate Drug D from consideration and focus their attention on Drugs A and B using the revised Dashboard shown below:

The tradeoffs between these two drugs are now clearly illustrated. Drug A is less expensive and safer but not quite as effective and more difficult to use. For this patient, they decide the effectiveness and side effects differences are too small to matter. The choice of treatment then comes down to whether cost or ease of use is more important. If cost is more important the better choice is Drug A. If ease of use is more important, the better choice is Drug B.

Musings

The dashboard I used for this illustration was built using Apple Numbers, a simple spreadsheet program. Adding and removing options only requires checking or unchecking a box. The process described in the illustration sounds tedious but flows quickly during a real application. In the first study I did with medical decision dashboards, the study subjects were able to analyze a dashboard comparing nine options over five criteria and choose a preferred treatment in an average of 4.6 minutes. [3]

Information visualization, particularly using modern, widely available interactive computer graphics, is a powerful tool for presenting information and using it to guide decisions. Because the most important skills involved are pre-attentive visual attributes that do not need to be learned, information visualization is an ideal way to help people, particularly those with limited numeracy skills, become better informed and engaged in choices affecting their health. Seems like a wonderful way to promote informed, patient-centered decision making in clinical practice settings. We should start using it.

References

1. Quotation from Card SK, Mackinlay JD, and Shneiderman, B. (1999) Readings in Information Visualization: Using Vision to Think. Academic Press included in Stephen Few. Now you see it. An introduction to visual data sensemaking. Second Edition. El Dorado Hills, CA: Analytics Press; 2020.

2. Dashboards: Making Charts and Graphs Easier to Understand [Internet]. [cited 2023 Feb 14]. Available from: https://www.nngroup.com/articles/dashboards-preattentive/

3. Dolan JG, Veazie PJ, Russ AJ. Development and initial evaluation of a treatment decision dashboard. BMC Med Inform Decis Mak. 2013 Dec;13(1):51.

Information visualization

A tool for adapting deliberative thinking for use in busy clinical settings

Modern data graphics can do much more than simply substitute for small statistical tables. At their best, graphics are instruments for reasoning about quantitative information. Often the most effective way to describe, explore, and summarize a set of numbers … is to look at pictures of those numbers. ~ Edward Tufte [1]

---

In the February 10, 2023 Musing, I cited research from sports decision science studies suggesting that it is possible to use deliberative decision methods in time-limited situations if they are appropriately modified to fit the context.

In the past two Musings, I’ve proposed that constructing a simple decision model is a quick and easy way to incorporate deliberative decision making into busy clinical settings.

Another simple way to adapt deliberative decision making for use in busy clinical settings is data visualization.

Clinical decisions typically involve comparing the expected outcomes of alternative courses of action across several dimensions that can be expressed quantitatively such as effectiveness, risk of side effects, and cost. Informed decision making involves making comparisons among the options on these measures based on an accurate understanding of the differences between them.

Comparative data can be presented in several ways. Qualitative terms such as rare or common are frequently used, but lack precision and can be easily misunderstood. Tabular displays of data are also common, but take time to interpret and can result in information overload if too many discrete data points are included. Presenting the information visually overcomes both of these problems.

In the words of Stephen Few:

When we represent quantitative data in visual form, our ability to think about it is dramatically enhanced. Visual representations not only make the patterns and relationships among numbers visible, they also extend the capacity of our memory, making available to our eyes what we could not otherwise hold in our minds simultaneously. Simply stated, data visualization augments our ability to think quantitatively. [2]

Visual data formats work by taking advantage of features of visual images that humans perceive automatically at a subconscious level. These features are called preattentive visual attributes. They have been grouped into four categories: form (such as length and width), color, spatial position, and motion. The two best attributes to precisely show quantitative values are length and two dimensional position.

Graphics that take advantage of the appropriate preattentive attributes expand our ability to think quantitatively by using our visual cortex to supplement and expand our limited working memory capabilities. The result, per Stephen Few is:

“When quantitative values are displayed as visual images that exhibit meaningful patterns, more information is chunked together … so we can think about a great deal more information simultaneously than if we were relying on tables of numbers alone.” [2]

For example, compare the following table and graphs – both contain the same data:

Which format makes it easier to compare the response rates of all 4 drugs quickly? The answer is the graph, which allows one to compare the data easily with a simple glance. The table, on the other hand, forces one to keep the individual values in mind and calculate the differences while making the comparisons.

One could argue that the ease of comparing the four drugs in this example is not that different between the table and the graph. However, the impact of graphic displays are especially time saving when multiple comparisons are required. For example,

consider which of the four drugs to choose when risk of side effects and cost are added to the mix:

In this case, it is much easier and faster to make drug-drug comparisons using the graphs than the table.

Musings

Access to pertinent up-to-date information is an important part of quality decision making. However, in addition to access, good decisions require accurate understanding of the information and the ability to use it to help make a choice. In time-constrained clinical settings, the ability to have the information readily at hand in a format that can be interpreted and understood quickly and easily is even more important. Graphic data displays greatly facilitate this process.

References

1. Edward R. Tufte. The visual display of quantitative information. Graphics Press, 1983 as quoted in Stephen Few, Now you see it. [2]
2. Stephen Few. Now you see it. An introduction to visual data sensemaking. Second. El Dorado Hills, CA: Analytics Press; 2020.