--- title: 社内勉強会_SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model tags: #humanoid #reinforcement learning #model-based rl #whole-body control #unitree #unitree g1 #sim-to-real #corl 2026 #bptt #mujoco author: [NABLAS株式会社](https://docswell.com/user/nablas) site: [Docswell](https://www.docswell.com/) thumbnail: https://bcdn.docswell.com/page/V7P39WVXJ8.jpg?width=480 description: 本スライドでは、人型ロボットの全身トラッキングにおいて、クラスタ規模の強化学習を単一GPU規模へと劇的に効率化した「SuperMimic」の論文を解説しています。 従来、ヒューマノイドの多様な全身動作追従には膨大な計算資源（SONIC: 128×L40で7日間＝計21,504 GPU時間）を要するPPOが主流でした。本研究では、単一の微分可能な世界モデル（Differentiable World Model）を学習し、BPTT（時間逆伝播）を通じて解析的勾配を活用することで、わずか「1×H100・3時間」でSONIC同等以上のリターンを達成。さらに約10時間の学習でベンチマーク上の失敗ケース0件（SONICは179件）を達成し、実機Unitree G1へのゼロショット転移にも成功しています。 In this presentation, we explore "SuperMimic," a paper that drastically compresses cluster-scale reinforcement learning down to a single GPU for humanoid whole-body motion tracking. While cluster-scale model-free PPO pipelines (such as SONIC, requiring 128× L40 GPUs for 7 days / 21,504 GPU-hours) have long dominated humanoid whole-body control, computational demand has remained a critical bottleneck to scaling. SuperMimic overcomes this barrier by learning a single differentiable world model and leveraging analytic gradients via Backpropagation-Through-Time (BPTT). It surpasses the return of SONIC in just 3 hours on a single H100 GPU (>1000× compute reduction). Furthermore, after ~10 GPU-hours of training, it achieves zero failures across 129,514 benchmark motions (compared to 179 failures for SONIC) and successfully transfers to a physical Unitree G1 humanoid robot. published: November 09, 26 canonical: https://docswell.com/s/nablas/59NQ4V-2026-11-09-155059 --- # Page. 1 ![Page Image](https://bcdn.docswell.com/page/V7P39WVXJ8.jpg) Paper Discussion SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model Liao et al. · UC Berkeley · CoRL 2026 NABLAS Research/Engineering Community # Page. 2 ![Page Image](https://bcdn.docswell.com/page/2JV4M8Z37Q.jpg) TL;DR One world model makes whole-body tracking efficient SuperMimic uses differentiable imagined rollouts to train a whole-body tracker that exceeds SONIC’s return in 3 hours on one GPU. Question Key idea Finding Can whole-body tracking Learn one dynamics model One H100, 3 hours to scale without cluster-scale across motions, then exceed SONIC’s return. The PPO training? backpropagate the controller also transfers to tracking return through it. a real Unitree G1. Liao et al., CoRL 2026 · Abstract and §4.1 © NABLAS Inc. All Rights Reserved 2 # Page. 3 ![Page Image](https://bcdn.docswell.com/page/5EG1G54YJL.jpg) Paper SuperMimic studies scalable humanoid motion tracking Title SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model Authors Qiayuan Liao, Yiyang Shao, Junfeng Long, Kevin Zakka, Roberto Horowitz, Pieter Abbeel, Koushil Sreenath Affiliation UC Berkeley Venue Conference on Robot Learning (CoRL), 2026 Paper / project arxiv.org/abs/2610.28480 · supermimic.github.io Paper type Method and system, with efficiency analysis and real-robot transfer Focus • A direct comparison of model-free policy learning and analytic gradients through shared dynamics. Paper title page and project citation © NABLAS Inc. All Rights Reserved 3 # Page. 4 ![Page Image](https://bcdn.docswell.com/page/4JQDXZG67P.jpg) SONIC connects motion inputs to humanoid control Background • Reference: SONIC’s motion encoders and shared control interface • Our comparison focuses on how the motion-tracking controller is trained SONIC overview figure retained from the source deck © NABLAS Inc. All Rights Reserved 4 # Page. 5 ![Page Image](https://bcdn.docswell.com/page/K74Z635ME1.jpg) Key idea BPTT learns through predicted action consequences • PPO: estimate policy updates from sampled returns and advantages • BPTT: differentiate imagined return through the learned dynamics • MuJoCo supplies transitions. Gradients pass through the world model Intuition Shared body dynamics let one model reuse what it learns across many different reference motions. Source-deck schematic · SuperMimic §3.2‒3.3 © NABLAS Inc. All Rights Reserved 5 # Page. 6 ![Page Image](https://bcdn.docswell.com/page/LJ1RV1WYJG.jpg) Method 1/2 Data collection, model fitting and policy updates repeat • Collect transitions in MuJoCo Warp and store them in a replay buffer • Fit the world model, then improve the policy through imagined rollouts Intuition Refreshing the model as the policy changes keeps training tied to the states the policy actually visits. Liao et al., CoRL 2026 · Figure 2 and §3.1 © NABLAS Inc. All Rights Reserved 6 # Page. 7 ![Page Image](https://bcdn.docswell.com/page/GJW1L861E2.jpg) Method 2/2 Eight-step model fitting supports 32-step imagination STEP 1 STEP 2 STEP 3 Collect simulator transitions Fit 8-step predictions Improve over 32 steps Train the world model on short Hold model weights fixed, Store state, executed action autoregressive windows from update the policy with BPTT, and simulator context in the the buffer. and bootstrap with a critic. replay buffer. Intuition Short imagined rollouts limit the prediction horizon. The critic supplies value beyond those 32 steps. Liao et al., CoRL 2026 · §3.2‒3.3 © NABLAS Inc. All Rights Reserved 7 # Page. 8 ![Page Image](https://bcdn.docswell.com/page/4EZPM8YXJ3.jpg) Evidence 1/3 SuperMimic exceeds SONIC’s return in 3 hours Training / evaluation SONIC SuperMimic SuperMimic GPU hardware 128 × L40 1 × H100 2 × H100 Training time 168 h 3h 9.8 h Total compute 21,504 GPU-h 3 GPU-h 19.6 GPU-h Normalized return ↑ 0.90 >0.90 >0.90 Failures out of 129,514 ↓ 179 25 0 Scope Hardware and training data differ. The failure benchmark measures coverage of training motions. Liao et al., CoRL 2026 · §4.1, Figure 3(a), Table A.8 © NABLAS Inc. All Rights Reserved 8 # Page. 9 ![Page Image](https://bcdn.docswell.com/page/Y76MNPDPEV.jpg) Fast rollouts and analytic gradients both contribute >10× ≈64× Faster rollout throughput Fewer imagined steps Evidence 2/3 • Left: world model versus MuJoCo Warp on H100 • Right: reach the best PPO-WM training return on LAFAN1 locomotion Liao et al., CoRL 2026 · Figure 5 and §4.2 © NABLAS Inc. All Rights Reserved 9 # Page. 10 ![Page Image](https://bcdn.docswell.com/page/G75Z5K3PJ4.jpg) One tracker transfers to a physical Unitree G1 Evidence 3/3 • The paper demonstrates diverse whole-body and acrobatic motions • One general controller, without per-motion tuning Liao et al., CoRL 2026 · Figure 6 and §4.4 © NABLAS Inc. All Rights Reserved 10 # Page. 11 ![Page Image](https://bcdn.docswell.com/page/9J2RNWZZ7R.jpg) Terrain, real dynamics and joint interfaces remain open Limitations Interfaces The preliminary velocity study updates only its encoder. The benefit of joint decoder updates remains untested. Model The world model learns only from simulator transitions. Real-robot data and system identification remain future work. Contact Training uses flat ground. Terrain and object interaction remain future work. Author-stated limitations · Liao et al., §6 © NABLAS Inc. All Rights Reserved 11 # Page. 12 ![Page Image](https://bcdn.docswell.com/page/DEYDPLRNEM.jpg) Takeaway training Input: Human mo-cap dataset policy output: humanoid joint positions policy training: - SONIC: PPO, in Mujoco - SuperMimic: BPTT, with a learned dynamic model results: 3 gpu hours as the same performance of 168h*128 gpus for SONIC 10 gpu hours achieved zero failure case (179 for SONIC) They hosted the demo in the project page, play with it: https://supermimic.github.io/demo/ © NABLAS Inc. All Rights Reserved 12 # Page. 13 ![Page Image](https://bcdn.docswell.com/page/VJN654DVJ8.jpg) THANK YOU WEBSITE SOCIAL MEDIA https://www.nablas.com/ Paper & demo: supermimic.github.io