AI in Autonomous Vehicles

Abstract

Autonomous vehicles represent one of the most demanding AI applications, requiring real-time perception, prediction, and planning in safety-critical open-world environments where failure modes can be fatal. This study presents a controlled evaluation of six perception-prediction-planning architectures -- modular pipeline (detect-track-predict-plan), end-to-end learned (sensor-to-control), hybrid modular-learned, transformer-based BEV perception (BEVFormer), world-model planning (MILE), and neuro-symbolic reasoning -- across four autonomous driving benchmarks: nuScenes 3D detection and tracking, Waymo Open motion prediction, CARLA closed-loop driving simulation, and the long-tail corner case suite (CODA). A total of 2,160 experiments were conducted. BEVFormer achieved the highest nuScenes detection NDS (56.8 +- 0.4) by fusing multi-camera views into a unified bird's-eye-view representation through spatial cross-attention. The world-model planner (MILE) achieved the highest CARLA driving score (78.4 +- 2.8) by learning a predictive model of scene evolution and planning trajectories through imagination. The modular pipeline achieved the best interpretability and the highest corner-case detection rate (68.4% on CODA vs. 42.6% for end-to-end), because its explicit intermediate representations (detected objects, tracked trajectories) enable targeted failure analysis. End-to-end models achieved higher average driving performance (72.8 vs. 68.4 driving score) but showed 2.8x higher catastrophic failure rate on long-tail scenarios. The sensor suite analysis revealed that LiDAR-camera fusion outperformed camera-only by 8.4 NDS points but the gap narrowed from 12.6 (2021) to 8.4 (2023) as camera-only methods improved. A practical AV architecture selection guide mapping safety requirements, sensor configuration, and compute budget to recommended approaches is proposed.