Artificial intelligence has the potential to transform health and medicine, from improving access to mental health care to tagging molecular details in tumor images. AI algorithms can spot patterns in massive troves of data -- even better than humans, in some cases.
These technologies, however, are susceptible to bias, Stanford researchers describe in a recent review article in EBioMedicine. Biases in sex, gender and race within datasets can produce models that are poor predictors for certain groups, and the use of such biased models could exacerbate existing structural health disparities, they wrote.
"The white body and the male body have long been the norm in medicine, guiding drug discovery, treatments and standards of care," said Londa Schiebinger, PhD, an article co-author and professor of history of science. "It's important that we do not let AI devices fall into that historical pattern."
As the use of AI in health care continues to expand, challenges of bias will only grow. In the article, the researchers provide short- and long-term approaches for addressing AI bias.
"We want to make sure these technologies have broad benefits for diverse demographics and populations," said James Zou, PhD, a coauthor with Schiebinger. Zou is professor of biomedical data science.
Examples of bias
The researchers point to data gathered from pulse oximeters as an example of potential bias within artificial intelligence. A pulse oximeter shines a light through skin to measure the amount of oxygen circulating in a patient's red blood cells -- based on light absorption.
But melanin, a pigment that gives skin its color, can affect readings because it also absorbs light. A recent study found that Black patients are three times more likely to have low oxygen levels missed by pulse oximeters than white patients. Oximeters also misstate oxygen levels more often in women than in men.
"The pulse oximeter is an instructive example of how developing a medical technology without varied demographic data collection can lead to biased measurements and thus poorer patient outcomes," said Zou. These biases could sabotage algorithms that use pulse oximeter data to guide treatment decisions.
Another application of AI is the use of an individual's biomarkers, such as their microbiome, to provide recommendations for reducing future disease risk. But there are microbiome variations between different ethnic groups, so how well such predictive models work for broad audiences requires careful evaluation.
Changes for the better
To reduce biases in the short term, Zou and Schiebinger wrote, datasets used for developing and evaluating algorithms should include more diversity. For example, images depicting certain skin conditions on darker skin tones could improve dermatological models. And policy changes by regulatory and funding agencies could lead to longer term structural solutions, in addition to helping guide the direction of research, they wrote.
"These agencies have a great role to play because they are distributing taxpayer money, which means that the funded research must benefit all people across the whole of society," Schiebinger said. The National Institutes of Health has made strides, requiring in January 2016 that researchers consider sex as a biological variable in clinical studies. Schiebinger anticipates the NIH developing similar policies for gender, as well as race and ethnicity.
Medical journals could also insist that manuscripts require analysis by sex, gender or both where appropriate prior to acceptance, they noted.
"We're hoping to engage the AI biomedical community in preventing bias and creating equity in the initial design of research, rather than having to fix things after the fact," Schiebinger said.
This is a condensed version of story from the Stanford School of Humanities and Sciences. Read the full story here
By Wirestock
