Neural Architecture Search Automation

Abstract

Neural architecture search automates the design of deep learning architectures by searching over a space of possible configurations to find those that optimise a target objective, replacing the intuition-driven manual design process with systematic optimisation. This study presents a controlled comparison of six NAS methods -- random search (baseline), reinforcement learning-based (ENAS), evolutionary (RegularizedEvolution), differentiable (DARTS), one-shot supernet (OFA), and predictor-based (BANANAS) -- across four search spaces and tasks: CIFAR-10/100 cell-based search (NAS-Bench-201), ImageNet mobile-constrained search (OFA space), NLP architecture search (FlexiBERT), and hardware-aware search targeting Jetson Nano latency constraints. A total of 1,800 experiments were conducted under standardised search budgets (measured in GPU-hours rather than number of architectures evaluated, ensuring fair comparison across methods with different per-evaluation costs). One-shot methods (OFA) achieved the best accuracy-efficiency trade-off, finding architectures within 0.3% of the global optimum in NAS-Bench-201 at 50x lower search cost than RL-based methods. Differentiable search (DARTS) found competitive architectures rapidly but exhibited instability in 18.4% of runs, producing degenerate architectures dominated by skip connections. Predictor-based search (BANANAS) achieved the highest sample efficiency (fewest architectures evaluated to reach 90% of optimal), making it ideal for expensive-to-evaluate search spaces. Random search with early stopping, despite its simplicity, outperformed RL and evolutionary methods when given equivalent compute budgets on NAS-Bench-201, confirming that many search spaces are sufficiently benign that sophisticated search algorithms provide limited advantage over intelligent random sampling. A practical NAS deployment guide mapping search space complexity, compute budget, and hardware constraints to recommended methods is proposed.