TL;DR
• 20,000 hours of teleoperated manipulation data from 9 dual-arm robot configurations (AgiBot G1, AgileX, Galaxea R1Pro, Realman, ARX Lift2, Bimanual Franka, and others)
• Videos manually segmented into atomic actions, then labeled with global and sub-task descriptions via VLM
• GM-100 benchmark: 100 tasks × 3 platforms × 130 episodes per task = 39,000 expert demonstrations for post-training evaluation
• Full code, base model weights, and benchmark data released
• Paper: arXiv:2601.18692
• Code: github.com/robbyant/lingbot-vla
• Models/Data: HuggingFace collection
What’s in the data
Each of the 9 embodiments has a dual-arm setup with multiple RGB-D cameras (typically 3 views: head + two wrists). The raw trajectories were collected via teleoperation (VR-based or isomorphic arms depending on the platform). Action spaces range from 12-DoF to 16-DoF depending on the robot. Every video was manually segmented into atomic action clips by human annotators, with static frames at episode start/end removed. Task and sub-task language instructions were then generated using Qwen3-VL-235B. An automated filtering pass removes episodes with technical anomalies, followed by manual review using synchronized multi-view video.
The data curation pipeline is probably the part I found most interesting to work through. About 50% of the atomic actions in the test set are absent from the top 100 most frequent training actions, which gives a sense of how much distribution shift the benchmark actually tests.
Benchmark structure
The GM-100 benchmark covers 100 tabletop manipulation tasks evaluated on 3 platforms (AgileX, AgiBot G1, Galaxea R1Pro). Each task gets 150 raw trajectories collected, top 130 retained after quality filtering. Object poses are randomized per trajectory. Evaluation uses two metrics: Success Rate (binary task completion within 3 minutes) and Progress Score (partial credit based on sequential subtask checkpoints). All evaluation rollouts are recorded in rosbag format and will be released.
For context on the numbers: LingBot-VLA w/ depth hits 17.30% average SR and 35.41% PS across all three platforms. π0.5 gets 13.02% SR / 27.65% PS on the same tasks with the same post-training data. These are not high numbers in absolute terms, which honestly reflects how hard 100 diverse real-world manipulation tasks actually are.
Scaling observations from the data
One thing worth flagging for people interested in data scaling: going from 3,000 to 20,000 hours of pre-training data showed consistent improvement with no saturation. The per-platform curves (Fig 5 in the paper) all trend upward at the 20k mark. This is on real hardware, not sim, which makes the continued scaling somewhat surprising given how noisy real-world data tends to be.
Training codebase
The released codebase achieves 261 samples/sec/GPU on an 8-GPU setup (1.5x to 2.8x over OpenPI/StarVLA/Dexbotic depending on the VLM backbone). Uses FSDP with hybrid sharding for the action expert modules and FlexAttention for the sparse multimodal fusion. Scaling efficiency stays close to linear up to 256 GPUs.
Caveats
All data is dual-arm tabletop manipulation only. No mobile manipulation, no single-arm, no legged locomotion. The 17% average success rate means these tasks are far from solved. Depth integration helps on some platforms more than others (AgileX benefits most, AgiBot G1 barely moves). The language annotations are VLM-generated after manual segmentation, so annotation quality depends on both the human segmentation and the VLM’s captioning accuracy.
Disclosure: this is from Robbyant. Sharing because 20k hours of labeled real-robot data with a standardized benchmark is something I haven’t seen at this scale in an open release before, and the benchmark data alone could be useful for people working on evaluation protocols for embodied AI.
Curious what formats and subsets would be most useful for people here to work with directly.
submitted by /u/Independent_Plum_489
[link] [comments]
