Understanding the decision-making processes of physicians through simulation and games.

Introduction 

One of the most fundamental challenges in healthcare market research is understanding the decision-making processes of physicians. As those of us in the industry know, this is no trivial exercise. Like other research respondents, physicians are often all too ready to give the “correct” answer regarding why they do what they do. In fact, by virtue of their extensive training and elevated place within society, they are even more prone to do this than lay consumers. Moreover, they are not immune to the foibles of memory that plague all respondents. 

Delving deeply into physician decision-making is further hampered by the nature of our research techniques—both quantitative and qualitative. On the quantitative front, surveys involving repetitive reporting of patient counts, treatment allocations, prescribing intent, attitudes, beliefs and behaviors tax both the cognitive abilities and patience of physician respondents. The result is lower physician engagement in surveys, as well as poor response and completion rates. While the qualitative side of the equation fares somewhat better, it is still difficult for interviewers to put the physician in the moment of diagnosis or treatment in order to understand how they think and behave in the real world. An under-appreciated approach to addressing these challenges is found in software-based simulation. The fundamental idea is this. Instead of asking physicians what they do and why, simulation enables us to place physicians into a digital environment and observe how they diagnose and treat virtual patients over multiple visits and over the course of disease progression. Everything that physicians do in such a simulation environment can be recorded, uncovering not only diagnosis and treatment behaviors, but the decision-making process itself—as revealed through analysis of the information accessed and utilized by the physicians to make their decisions. 

Simulation versus Games

For most market researchers, the word “simulation” is likely to suggest familiar research techniques such as conjoint, or more complex and less commonly deployed models such as systems dynamics. Others might imagine some modest version of virtual reality technology. But any spontaneous association between simulation and games is unlikely to come to mind. In other quarters, the close relationship between simulation and games is well recognized—so much so that experts in simulation and game design have struggled to develop widely accepted definitions that clearly distinguish the two. All agree, however, that simulations and games share a great deal of DNA. Exploration of this shared DNA provides critical guidance for the creation of simulations that “hack” the brains of respondents and reveal insights into behavior and thinking that are often missed in traditional questionnaires or qualitative.

So what are the relevant components of this shared DNA? Our experience in developing InTask™, a market research simulation inspired by game design, provides some answers. First, both simulations and games offer “branched narratives”—stories whose ending is unclear at the start. As a result, they create a degree of interest and curiosity in the participant about the outcome of their virtual tasks within the research. Moreover, in both simulations and games, the participants are imbued with “agency”—the ability to impact the narrative based on their actions, and to learn the consequences of those actions. This further engages them in the research process. Finally, both simulation and games place participants in a malleable environment that can be designed with a particular purpose in mind. In casual games, the ultimate purpose may simply be to provide entertainment. In market research centric simulations, the purpose might be to understand how physician actions change in the face of differing patient presentations, changes in access to medications, availability of new products, or new information regarding medications or treatment guidelines.

The distinction between simulation and games on the one hand, and gamification on the other is vitally important. In our industry, gamification refers to the selective borrowing of game mechanics to enhance the engagement of respondents within market research surveys. Gamification has not generally been understood as leveraging the essence of what makes games appealing—namely, branching narratives,agency, consequence and malleable environments. As a result, we sometimes sprinkle a bit of gamification on surveys in hope of making traditional survey questions more palatable. For example, a gamified challenge like, “How many brands of anti-hypertensive medications can you name in 10 seconds?” might replace a traditional brand awareness question. True enough, gamified approaches like this can make traditional survey tasks more interesting and increase survey completion rates. But it is not clear that they bring us closer to accurately capturing physician behavior or revealing the physician mindset. This is one of the reasons why the early promise of gamification in physician market research has not met the expectations of many healthcare researchers. Another and more insidious reason, perhaps, is that our cautious healthcare industry treads with great caution on anything that might suggest to physicians that a survey task is not “serious”.

Herein lies the beauty of simulation. As we have demonstrated in our R&D for InTask™, simulation can leverage the essential appeal of games without appearing to be a game. The window-dressing of gamification is unnecessary because well-designed simulations are inherently engaging. In a quantitative pilot test of the InTask™ platform, we have shown that 9 out of 10 psychiatrists found the InTask™ simulation more interesting than the on-line surveys they typically encounter. In openended responses from this same pilot, many even said that the simulation was “fun”. Beyond engagement, we have also demonstrated that simulation provides a very flexible platform for experimentation. Just as in “The Matrix”, we can manipulate the simulation environment to emulate specific situations of interest to researchers, without drawing undue attention the experiment itself. One way we have done this is by creating pairs of virtual patient “clones” to evaluate how targeted differences in patient presentation affect physician prescribing. In a nod to the flexible experimentation approaches popularized by Silicon Valley, we call this approach “A/B testing”. 

Quantitative and Qualitative Use of Simulation

The creation of simulations that can replace or complement questionnaires in many quantitative market research applications is game-changing. Treatment algorithm, patient flow, physician segmentation, forecasting, TPP and message testing are all appropriate applications for simulation. And in all these applications, the intuitive simplicity of a well-designed simulation interface helps physicians spend more time thinking about what they do than about what a survey question means. The benefit is better data, generated quickly and cost-effectively through physician engagement in a process that is far more evocative of the real-world than a questionnaire. 

But the merits of simulation are not limited to quantitative market research. In qualitative research as well, simulation can play a key role in bringing physician decision-making into focus. When virtual patient based simulations are utilized in qualitative—particularly in combination with a technique known as cognitive interviewing, researchers are better able to penetrate generalities and reveal the actual cognitive processes of physicians. 

Cognitive interviewing, sometimes referred to as “think out loud”, refers to having individuals verbalize their thinking process in real time as they engage in an actual task. The technique is frequently used in educational circles and in user experience testing. It has not been used extensively with physicians given the impracticality of having physicians talk about their thought processes with an interviewer as they examine an actual patient. The closest approximation we have to cognitive interviewing in qualitative is the review of patient records, brought into the interview by the physician at the recruiter’s request. While useful, this technique suffers from at least two significant drawbacks. First, the treatment decision captured in the patient record represents a past event. This can trigger post-hoc rationalization of prior acts that preserves the appearance of a considered, thorough, evidence-based and socially respectable thought process. A second and related drawback is that diaries often fail to place the physician firmly at the point in which outcomes are uncertain and a range of possible diagnoses and treatment options may be on the table. Since the future is known, the physician’s reported thinking process tends to traverse a straight line to their selected action, often ignoring the exploration of alternatives they may have considered.

An excerpt from an actual interview illustrates the richer insights into physician decision-making which can be gleaned from simulation supported cognitive interviewing. Here, a psychiatrist speaks about his thought process as he reviews information in InTask™ about a virtual patient suffering from schizophrenia: 

“I’m going to go back to the patient because I want to review her history. She’s currently on Haldol and Lexapro but she’s quite, she has a lot of positive symptoms of psychosis and these are the symptoms that are typically most responsive to anti-psychotic medication. So I have to assume that either the dosage of medicine she is on is inadequate, or she is non-compliant with medicine, or the medicine is just not working for her so I have a couple of different possibilities there that are going through my mind…” 

“I’ve reached the conclusion now that the problem is non-compliance. We have a medication that has helped her in the past so we want to try giving it to her and let’s say he (the caregiver) comes back in a week or something, says he’s giving it to her orally and she does indeed, she is indeed somewhat better. Then okay, I would propose to them that we give this as a once monthly injection. Let’s give the decanoate as, let’s try Haldol decanoate, that way we don’t have to take it every day.”

It is worthwhile noting that the physician is speaking in the present tense. He is in the moment of dealing with a problematic patient. He is also invested in the task, knowing that his actions will impact the outcome of the patient in the simulation and that he will have to deal with that outcome when the virtual patient returns. His thinking is revealed under conditions of uncertainty and risk—a critical aspect of real-world decision-making that is lost in traditional research techniques. This is how simulation “hacks” the brains of physicians. By dint of training and years of practice, physicians become adept at solving puzzles—puzzles that take the form of challenging patients, difficult diagnoses, or risky treatment decisions. Simulations like InTask™ tap into the physician’s mental programming and heuristics by presenting puzzles in the form of virtual patients—from routine to challenging, depending on the research objective. The result is greater physician engagement with the research process. As one physician participant in our pilot tests said, “We love stuff like this… This works for our brain because this is how we think… we think, we assess, we treat, we reassess, we treat, we see if our diagnosis was right, we reassess, we treat and that’s how it works.” 

Conclusions 

The use of simulation in healthcare market research is not new. But it has largely been boxed in and narrowly applied to a small range of applications such as forecasting. In a similar way, gamification—as generally understood in the research industry, has dimmed our awareness of the essential appeal of games. When we combine a more expansive vision of simulation with the underlying allure of games, we discover a new and exciting tool that can address some of the shortcomings of our current methods, drive better engagement with the research process, and deliver better insights into physician decision-making.