The whole robot benchmark, fully loaded.

Many benchmarks exist from both hardware vendors and community members, but nearly all focus on evaluating a particular component or algorithm in isolation. Often times, multiple of these algorithms need to be run together which may interact negatively when composed into a full system due to sharing of limited CPU/power or conflicts accessing accelerated computing resources (GPU, NPU, FPGA, etc).

This project fills the gap with a reproducible, independent benchmark for comparing compute solutions for robotics or physical AI applications in all of their complexity. A platform that wins every microbenchmark can still fail to hold a 30 Hz control loop once planning, perception, sensor drivers, and a VLM are all running simultaneously on it at once. That interaction and potential failure mode is what this benchmark measures and seeks to expose.

Nav2 autonomously navigates a forklift material handling robot within a 200,000 sqft (18,600 m²) industrial warehouse. It moves pallets from shipping/receiving to shelving units while processing multiple 3D lidars, 2D safety lidars, RGBD cameras, and internal sensors. This workload is representative of dozens of companies and tens of thousands of robots deployed today in production environments.

Alongside it, Gemma 4.0, a popular VLM, exercises the platform's GPU. It is integrated into the navigation behavior tree for scene understanding, so its answers actually affect the robot's decisions.

Full methodology →