Based on an Open Access study published in Lancet Digital Health, AI has an unexplainable ability to discern race from image (MIMIC-CXR, CheXpert, National Lung Cancer Screening Trial, RSNA Pulmonary Embolism CT, and Digital Hand Atlas) datasets while human radiologists cannot. “The results from our study emphasise that the ability of AI deep learning models to predict self-reported race is itself not the issue of importance. However, our finding that AI can accurately predict self-reported race, even from corrupted, cropped, and noised medical images, often when clinical experts cannot, creates an enormous risk for all model deployments in medical imaging”.
In discussion, the study reports “The regulatory environment in particular, while evolving, has not yet produced strong processes to guard against unexpected racial recognition by AI models; either to identify these capabilities in models or to mitigate the harms that might be caused.” The study continues, “We note that in the context of racial discrimination and bias, the vector of harm is not genetic ancestry but the social and cultural construct that of racial identity, which we have defined as the combination of external perceptions and self-identification of race. Indeed, biased decisions are not informed by genetic ancestry information, which is not directly available to medical decision makers in almost any plausible scenario. As such, self-reported race should be considered a strong proxy for racial identity.”
An independent journalist remarks “If the factors that enable models to predict race accurately are both very general and not obvious to humans, AI models that are currently being used could accidentally be perpetuating racial disparities in healthcare.”
What exactly is being said here? That AI will intentionally misdiagnose one patient over another because it’s inherently racist? That seems to attribute the deep learning capability of software with sentience because a human counterpart cannot duplicate the result. Perhaps it’s witchcraft? That some treating clinicians will not act on critical imaging results if they know the race of the patient? That seems like a very real problem that we should solve -right now.
It’s software. It’s not a seemingly “sentient” being like Alexa or Siri, which are gathering identity based data for marketing purposes, digging deep into your identity with every word you speak and sharing with their masters who know everything you buy and everywhere you go. And those two most certainly are aware with what race you identify through your buying habits, questions you ask and even the inflection of your voice.
If you want to remove race from the algorithm, fix the source code. It’s called machine learning for a reason. Dig deeper to find the reason the machine learns to be better at a task than it’s human counterpart. Perhaps after scanning 50 million images it has learned a thing or two.
Ok, this isn’t funny, it’s human health. According to the study, “Race and racial identity can be difficult attributes to quantify and study in health-care research and are often incorrectly conflated with biological concepts (eg, genetic ancestry). In this modelling study, we defined race as a social, political, and legal construct that relates to the interaction between external perceptions (ie, “how do others see me?”) and self-identification, and specifically make use of self-reported race of patients in all of our experiments. We variously use the terms race and racial identity to refer to this construct throughout this study”.
Oh. I thought we were talking about science. This is the kind of thinking that will muddy the actual science of incorporating the Social Determinants of Health (SDOH) as a critical tool in healthcare diagnosis and treatment. It is estimated that as much as 80% of a person’s health is determined by the social and economic conditions of their homes and communities and what nutrition and healthcare are available to them. Healthcare organizations pursuing value-based care must go beyond standard claims and medical data and integrate data sources that measure SDOH to effectively treat the whole person. Proliferating a study such as this creates a shadow over SDOH with the inference that self-identification somehow negates real-world science and is even dangerous. AI is racist!
The journalists who pick up a deeply flawed study issued without peer review and then run with it should not. The Lancet is an extremely respected medical journal, but there was that ‘hydroxychloroquine trials’ thing a couple years ago that forced them to change their editorial practices. This “science” is just playing too fast and loose for me.
I’ll probably be cancelled now.