Deep Reinforcement Learning in Robotics

Abstract

Deep reinforcement learning has demonstrated remarkable capabilities in simulated robotic tasks, yet deploying learned policies on physical robots remains fraught with challenges including sample inefficiency, reward specification difficulty, safety constraints, and the persistent sim-to-real transfer gap. This study presents a systematic evaluation of four deep RL training paradigms -- pure simulation training with domain randomisation, simulation-to-real fine-tuning, learning from demonstrations (DAgger and BC+RL), and direct real-world training with safety constraints -- across three robotic platforms: a 7-DOF Franka Emika manipulator (contact-rich grasping and insertion), a quadrupedal Unitree A1 robot (locomotion over varied terrain), and a mobile manipulation platform combining a Fetch base with a 7-DOF arm (navigation + pick-and-place in cluttered environments). A total of 1,680 experiments were conducted including 840 physical robot trials. Simulation training with domain randomisation achieved 78.4 +- 4.2% grasp success on the real Franka, while simulation-to-real fine-tuning with only 50 real-world episodes improved this to 89.2 +- 2.8% -- recovering 72% of the sim-to-real gap. Learning from 10 expert demonstrations (BC+RL) achieved 84.6 +- 3.4% grasp success with 5x fewer simulation steps than pure RL. For quadrupedal locomotion, domain-randomised policies transferred with only 6.8% velocity tracking error degradation, the smallest gap across all platforms. Direct real-world training with reward shaping and safety constraints achieved competitive manipulation performance (82.4 +- 4.8%) within 200 real episodes but required careful constraint design to prevent hardware damage. A practical deployment pipeline recommending training paradigm based on task contact complexity, demonstration availability, and acceptable real-world interaction budget is proposed.