Privacy-Preserving Deep Learning

Abstract

Deep learning's reliance on large datasets creates fundamental tension with privacy requirements: models trained on sensitive data can memorise and leak individual training examples through model outputs, gradients, or learned parameters. This study presents a controlled evaluation of five privacy-preserving deep learning approaches - differential privacy SGD (DP-SGD), federated learning with secure aggregation, homomorphic encryption for inference, knowledge distillation from private models (PATE), and synthetic data generation with privacy guarantees -- across four privacy-sensitive tasks: medical image classification (CheXpert chest X-ray), clinical NLP (MIMIC-III discharge summaries), financial fraud detection (credit card transactions), and recommendation systems (MovieLens-1M). Privacy was measured by formal epsilon-delta differential privacy guarantees and empirical membership inference attack success rate. A total of 2,040 experiments were conducted. DP-SGD at epsilon = 8 reduced accuracy by 4.8 +- 1.2% on medical imaging versus non-private training, while epsilon = 1 reduced accuracy by 12.4 +- 2.2%, confirming a substantial privacy-utility trade-off. PATE achieved the best trade-off: epsilon = 2 with only 3.4 +- 0.8% accuracy loss by transferring knowledge through noisy aggregation of teacher ensemble votes. Membership inference attack success dropped from 68.4% (non-private model) to near-random (52.8%) at epsilon = 4. Synthetic data generation preserved 86.4 +- 2.8% of downstream model utility while providing formal privacy guarantees. Federated learning with secure aggregation provided communication-level privacy but remained vulnerable to model-level attacks without additional DP noise. A practical privacy-preserving ML selection guide mapping privacy requirements, acceptable utility loss, and computational budget to recommended approaches is proposed.