The potential and pitfalls of AI in healthcare

Helping healthcare workers

Hospitals are facing a growing workforce crisis, with many healthcare professionals facing burnout. There are many different causes, but one important factor that is rarely taken into account is the administrative workload of physicians and nurses (such as charting, uploading electronic medical records, and tracking down lab results). All of these tasks can be overwhelming and leave physicians with less time to actually care for patients. This impacts not only healthcare workers’ job satisfaction but also the quality of care patients receive.¹

The strategic use of AI technology has the potential to alleviate this burden by streamlining administrative tasks, thus improving the quality of life for healthcare workers and freeing up more time for them to actually practice medicine. While AI may not yet be powerful enough to understand all the nuances a trained physician can, it can reliably automate or semi-automate chart review by summarizing patient information or generating clinical reports.^1,2,3 This not only reduces administrative workload but can also lower costs for patients and improve accuracy in billing, which can help prevent issues like insurance fraud.³

Improving patient outcomes

But AI’s potential extends well beyond administrative work—it has the potential to improve healthcare outcomes.⁴ For instance, its ability to interpret large amounts of data across many parameters can aid in making diagnoses. One AI model has even demonstrated an ability to calculate risk for breast cancer that is superior to traditional methods.^4,5 

The ability to synthesize large amounts of data can also be useful in designing treatments that are more tailored to individual patients, which could improve health-related outcomes and allow for more efficient use of resources. AI has even outperformed human doctors in treatment design when trained on a specialized condition.⁶ These advancements suggest a future where AI could enhance diagnostic and treatment processes beyond current human capabilities.

Granting greater accessibility

People around the world struggle to access healthcare. AI, particularly chatbots, promises greater accessibility by providing real-time information and advice. This is especially valuable for people in healthcare deserts where access to medical professionals is limited.^7,8 Chatbots could also support patients who may be reluctant to seek medical advice for a stigmatized condition such as an STD, substance abuse, or mental health issues.^8,9,10 Check out some of TDL’s previous research on technology to deliver mental health services, including a personalized mental health app and a digital mental health platform for students.

The prohibitive cost of healthcare is an additional barrier to many people seeking help. AI can reduce healthcare costs by replacing expensive imaging techniques and private consultations or eliminating the need to travel to a medical facility.¹¹ While AI is not a replacement for human doctors, it could serve as a valuable supplement to lower barriers to healthcare access.

Bias in, bias out

Although AI in healthcare holds lots of promise, there are warning signs that discourage implementation in its current state. For one, AI has to be trained on existing healthcare data to learn how to make decisions. In the instance of breast cancer diagnosis, an AI might be shown real mammograms along with the patient diagnosis to understand the relationship between the two and translate that into predictions for future data. 

But, if the dataset that an AI is trained on is biased, the AI will make biased decisions. A dataset might be biased because it does not include data from a balanced sample of participants (e.g., men and women) or because the results of the data have been based on biased human decision-making.^12,13 For demographic groups who historically and currently experience worse healthcare outcomes as a result of bias, such as gender or racial minorities, the training of AI on past care decisions could mean the reproduction of—or at worst, amplification of—existing systematic disparities.^14,15,16,17

This issue is not unique to machine learning. Prediction methods and guidelines in medicine have historically been built on health data collected disproportionately from white people and men. This often means that these methods are less accurate for women and people of color.^18,19 For AI to be beneficial for everyone, it needs to be able to predict reality—not just replicate biased human decision-making.

Data privacy concerns

Another significant issue that plagues the entire AI industry is data privacy. Many AI technologies are owned by private companies, and the implementation of AI in healthcare would require access to patients’ private health information.²⁰ This raises concerns about privacy breaches or potential conflicts of interest between private companies and patients. Ensuring the protection of sensitive health data is crucial as AI becomes more integrated into healthcare.

Reality check

Some machine learning algorithms have already been implemented in healthcare and there is evidence that the consequences are not just theoretical. A landmark study demonstrated that an algorithm used on millions of people a year in the U.S. to identify which patients would benefit the most from an extensive care program was racially biased. The algorithm used dollars spent on healthcare as a proxy for illness severity, leading it to conclude that white patients (who spent more on their healthcare) were sicker and in need of more care than Black patients, even if their risk scores were actually the same.¹²

Testing is best

The pitfalls of AI in healthcare don’t mean it should be abandoned. Rigorous testing can compare AI's decisions with human decision-making and real outcomes to ensure accuracy and minimize any bias.¹⁶  Software to test the accuracy and bias of algorithms already exists and more are in development. Consistent use of these tools could help prevent premature implementation of a biased AI.²¹

Regulation and legislation

Legislation often struggles to keep pace with innovation and AI in healthcare is no exception. Developing regulations and standards that promote equity in AI development and use in healthcare is essential.²² These regulations could include penalties for companies who develop or use AI that fails to meet an ethical standard to encourage accountability.¹⁸

Regulations also need to balance the inclusion of relevant sociodemographic factors like gender, race, or income in datasets used by AI, while preventing the misuse of this data and protecting individual privacy.¹⁸ In general, legislation needs to encourage the protection of patient data and the need for consent.²³

Meaningful collaboration

Bias in AI is not limited to training datasets; it litters the entire tech space, influencing who in society has a say in creating machine learning algorithms and what tools will be implemented in our healthcare system. Collaborations among private developers, the public sector, academia, and citizen co-designers are necessary to build equitable AI systems and ensure that these technologies serve everyone fairly.¹⁸

Anyone working in healthcare takes an oath to minimize harm; preventing the perpetuation of existing inequities with equitable and accurate AI can be part of that. But equity cannot be an afterthought—it is not good enough to implement biased technology and say that it will be fixed later.²² New technologies must be designed and implemented with ethics and human rights as central considerations, and with AI in healthcare, we have a golden opportunity to get it right the first time.²⁴