Algorithmic Bias

Algorithmic bias occurs when an AI system produces systematically less accurate or unfair outcomes for particular groups. In healthcare, it can arise when training data are unrepresentative of the population on which the system is used, causing the model to learn patterns that work well for overrepresented patients but poorly for others.

How unrepresentative data create bias

Healthcare AI learns statistical relationships from past records, images or measurements. If some populations or clinical conditions are inadequately represented, the learned relationships may not generalise to them.

• A dataset may underrepresent patients by sex, age, ethnicity, disability, geography or socioeconomic status.
• Data drawn from a few hospitals, devices or regions may reflect local practices rather than the wider patient population.
• Incomplete records and biased clinical labels can reproduce historical inequalities in diagnosis, access and treatment.
• A model may show good overall accuracy while concealing substantially poorer performance for a smaller subgroup.

Consequences in healthcare

Biased performance can affect both clinical safety and distributive justice because AI outputs may influence screening, diagnosis, prognosis and treatment decisions.

• Some groups may face more false negatives, leading to missed or delayed diagnosis.
• Other groups may face more false positives, causing unnecessary tests, anxiety or treatment.
• Unequal error rates can deepen existing health disparities and reduce trust in digital healthcare.
• Deployment in a population different from the training population can produce dataset shift, weakening reliability.

Reducing the bias

Bias control must cover the full AI lifecycle, from data collection to post-deployment monitoring.

• Developers should use sufficiently diverse, high-quality and context-appropriate datasets with lawful and ethical safeguards.
• Performance should be reported separately across relevant demographic and clinical subgroups, rather than only as an aggregate score.
• Models require external validation across hospitals, regions, devices and populations before wider use.
• Regular bias audits, human clinical oversight and post-deployment monitoring are necessary because patient populations and practices change.