Bias Detection in Machine Learning Systems

Abstract

Machine learning systems deployed in consequential domains increasingly exhibit measurable biases that produce disparate outcomes across demographic groups, yet the detection and measurement of these biases remains methodologically fragmented. This study presents a systematic evaluation of six bias detection approaches -- group fairness metrics (demographic parity, equalised odds, calibration), individual fairness testing, counterfactual fairness assessment, causal discovery of bias pathways, intersectional bias auditing, and temporal bias drift monitoring -- across four high-stakes ML applications: hiring recommendation (resume screening, 48K applications), credit scoring (Lending Club, 2.26M loans), facial recognition (FairFace, 108K images), and clinical risk prediction (MIMIC-III, 42K ICU stays). A total of 1,920 bias audits were conducted under standardised evaluation protocols. Group fairness metrics detected statistically significant disparities in all four domains: hiring models showed 12.4 +- 2.8 percentage point selection rate gaps between gender groups; credit models showed 8.6 +- 1.4 point false positive rate gaps between racial groups; facial recognition showed 14.8 +- 3.2 point accuracy gaps between skin-tone groups; clinical models showed 6.2 +- 1.8 point calibration gaps between racial groups. Intersectional auditing revealed that bias was 2.4x larger at intersections (e.g., dark-skinned women in facial recognition) than for any single protected attribute alone. Temporal monitoring detected bias drift in 34.2% of deployed models within 6 months, primarily from distribution shift in input data rather than model degradation. The choice of fairness metric substantially affected whether a system was judged fair: 42.8% of model-domain combinations satisfied demographic parity but violated equalised odds, confirming that metric selection is itself a consequential ethical decision. A practical bias auditing framework specifying metrics, intersectional scope, and monitoring frequency by deployment context is proposed.