A team of Stanford Medicine-based surgeons has paired up with a UC-Santa Cruz "serious game" developer and assistant professor, Edward Melcer, PhD, to create a game-based online platform for assessing surgical decision making.
The platform, called ENTRUST, is meant to provide a dynamic, interactive and standardized way to evaluate the readiness of surgeons-in-training to manage different types of cases, according to Dana Lin, MD, FACS, one of the platform's creators and a clinical assistant professor of surgery.
ENTRUST poses realistic patient scenarios through which a resident must navigate. The scenarios reflect domains of expertise known as entrustable professional activities, which are being developed by the American Board of Surgery.
"The game can reveal strengths and weaknesses even in rare types of cases that a surgical resident may not commonly come across during their training," said Cara Liebert, MD, FACS, a Stanford Medicine clinical assistant professor of surgery and co-creator of the platform.
Each session of the game begins with an illustration of a fictional patient sitting or lying on an examination table. The patient's vital signs are displayed overhead. A medical chart provides a brief synopsis of the patient's signs and symptoms. The player can "examine" the patient by passing a cursor over different parts of the patient's body.
The center of the screen features an interactive dashboard on which tests, images, and procedures can be ordered. Results of the diagnostic tests appear in the medical chart. Each decision the player makes is graded according to a carefully constructed scoring algorithm; points are lost for ordering unnecessary or harmful tests or procedures and gained for choices that lead to an accurate diagnosis or treatment. Neutral choices neither add nor detract points.
"Putting a tube in the chest of a virtual patient who doesn't need it loses a player significant points," said Lin, who is the associate program director of the general surgery residency program. "But if the patient suffers or dies because a player doesn't put a tube in her chest, the score takes a critical hit."
Upgrading to competency-based testing for new surgeons
"Most people think of a competent surgeon as one who has 'good hand' and is technically skilled in the operating room," said Lin. "However, the ability to make sound judgments regarding if and when to operate and what kind of operation should be done, is equally, if not more important."
Traditionally, oral exams are used to assess the surgical decision-making skills of graduating residents. However, oral examinations are costly, time-consuming, and resource intensive. ENTRUST poses one efficient way to meet that challenge for assessment of certain skills or knowledge in lieu of high-stakes exams.
Back in the game, depending on what's being evaluated, the player's virtual clinic may give way to a virtual hospital ward or operating room, where they must make more choices. The player is presented with questions that further explore how they would approach the case.
At a subsequent virtual clinic visit, they may need to manage post-operative complications. Decisions made in all these settings are entered into the scoring algorithm developed by surgeons and education experts to determine the players readiness to make such decisions on their own in the real world.
'Dynamic interaction'
LaDonna Kearse, MD, a third-year surgical resident at Howard University, currently spending two years at Stanford as a Surgical Education Fellow, used ENTRUST as part of a study of the platform conducted in May.
"I felt that it was assessing my ability to interact with a patient, not just my ability to answer hypothetical questions," Kearse said. "Instead of just multiple choice, you're having a dynamic interaction, you're examining the patient, their vitals, their chart. Compared to the kinds of tests we're used to, it also takes some of the stress off."
Lin, Liebert and their partners have developed and studied one type of module for ENTRUST, based on patients with groin hernias. Two studies tested a total of 62 players -- medical students, interns, surgical residents and more experienced surgeons -- and found that scores correlated closely with players' levels of experience, "demonstrating that ENTRUST is a robust assessment platform for surgical decision making," said Lin.
Lin and Liebert said that they'll be ready to roll out ENTRUST on a larger scale for evaluation of the full complement of American Board of Surgery entrustable professional activities by 2023.
The platform should have a profound effect on the way surgical knowledge and decision making are evaluated, but its influence could eventually extend well beyond that, said Liebert. It is so flexible and user-friendly that examiners can easily compose any kind of case they choose, said Lin.
"ENTRUST could help teach and evaluate clinical decision making in emergency medicine, anesthesia, pediatrics, nursing, or really any medical field," she said.
Photo courtesy of Dana Lin
