RESEARCH

Sim-to-Real Mobile-Robot Driving
with Unity ML-Agents

Period 2023.09 — 2024.04 Team 6 (80% contribution) Platform Clearpath Husky A200 · VLP-16

🎬 Real-robot driving 🎬 Curriculum learning

Built three Unity training environments (maze, static obstacles, real building corridor) with a LiDAR simulation matched to the actual VLP-16 spec.
Exported the trained policy to ONNX and deployed it on a physical robot via ROS; closed the sim-to-real gap with LiDAR noise injection and curriculum learning.
Achieved stable driving over a ~50 m straight corridor — about 6.7× the distance of the first attempt.

STEP 1Building the training environments

The Unity LiDAR was implemented to match the real Velodyne VLP-16 (channels, FOV, range) and its distances fed the RL state. Environments were built in order of difficulty: a maze, a static-obstacle course, and a replica of the lab building's corridor.

T-shaped maze training environment — Maze environment — robot (left) and goal (bottom right).

LiDAR point cloud visualized in Unity scene and game views — Simulated LiDAR point cloud (left: Scene view, right: Game view) with ROS connection info.

STEP 2Reward design and training analysis

Reward and termination were designed around LiDAR usage, distance, time, and collisions — and each change was analyzed through its training curves. Notably, a per-step penalty (−0.01) alone made the agent crash on purpose to end episodes early; switching to a 60-second episode time limit converged successfully while cutting training time by about 50%.

Cumulative reward curves comparing two reward designs — Training behavior by reward design — top: per-step penalty (intentional crashing, no convergence), bottom: time-limit design (converged).

STEP 3Sim-to-Real transfer

The policy was exported to ONNX and wrapped in a ROS node (Ubuntu 18.04, Melodic) controlling the physical Husky A200. Early runs destabilized quickly due to differences between simulated and real LiDAR; injecting Gaussian noise into the simulated LiDAR and applying curriculum learning (straight → intersection → real corridor) fixed it.

ROS node graph centered on the ONNX inference node — ROS control graph — /scan and /goal feed the ONNX inference node, which outputs velocity commands.

Curriculum environments from straight to intersection to corridor with driving logs — Curriculum learning environments (straight → intersection → lab corridor) and driving logs.

RESULT

Stable driving over a ~50 m straight corridor — about 6.7× the distance of the first attempt
Quantified the sim-to-real gains from noise injection + curriculum learning

DEMOVideos

Physical Husky A200 corridor driving (1:30)

Unity training montage — maze · obstacles · corridor

Husky A200 driving autonomously along a corridor — Husky A200 driving the corridor on the learned policy.

Sim-to-Real Mobile-Robot Drivingwith Unity ML-Agents

STEP 1Building the training environments

STEP 2Reward design and training analysis

STEP 3Sim-to-Real transfer

DEMOVideos

Sim-to-Real Mobile-Robot Driving
with Unity ML-Agents