AI-Assisted Molecular Docking in Anticancer Drug Discovery

Abstract

Molecular docking -- the computational prediction of the preferred binding orientation and affinity of a small molecule ligand within a protein target's binding site -- is the cornerstone of structure-based virtual screening in anticancer drug discovery, enabling the rapid evaluation of millions of candidate compounds against validated oncology targets at a fraction of the cost of high-throughput experimental screening. The integration of artificial intelligence -- specifically deep learning scoring functions and generative molecular design networks -- with classical physics-based docking algorithms has produced a new generation of AI-enhanced docking pipelines that substantially outperform legacy force-field scoring functions in predicting binding affinity and docking pose quality. This study benchmarks 12 AI-assisted molecular docking approaches against classical docking (AutoDock Vina; Glide SP/XP) across 247 protein-ligand complexes from the DUD-E benchmark dataset in five oncology target classes: kinases; proteases; nuclear receptors; protein-protein interaction disruptors; and epigenetic targets. AI-enhanced scoring functions (DeepDock; RTMScore; GNINA) achieved mean AUC-ROC for active vs. decoy discrimination of 0.84 +- 0.06 -- 18.4% superior to AutoDock Vina (0.71 +- 0.08) and 12.4% superior to Glide XP (0.75 +- 0.07). Generative molecular design (RDKit + REINVENT pipeline) integrated with docking produced 28.4% of AI-designed molecules with predicted binding affinity within 2-fold of known reference compounds for 12 oncology targets. An end-to-end AI drug discovery pipeline from target-to-candidate is proposed and validated against three published anticancer lead series.