← All writing
Progress Report · KTH RPL

Unitree G1 — Whole-body
Teleoperation & Data Collection

Platform
Unitree G1 (humanoid)
Window
Weeks 1–12
Status
VLA policy deployed
Completed Current direction Decision / transition Hardware risk Data collection
Goal Goal project north star

Humanoid whole-body training data on G1

Objective

Collect coordinated locomotion + manipulation demonstrations on the Unitree G1 to support future whole-body policy / VLA training.

01 · Input
Human VR motion
02 · Control
Whole-body controller
03 · Capture
Demonstration data
04 · Output
VLA / policy training
Week 1–2 Upper-body validation · sim & real

Upper-body VR teleoperation is working

Done
  • Deployed upper-body teleoperation using Meta Quest 2.
  • Validated in simulation and on the real G1.
Upper-body teleoperation in simulation
Simulation · upper-body tracking
Upper-body teleoperation on real G1
Real G1 · Meta Quest 2 input
Week 3 VLA scope expansion

Extend VLA to humanoid

Start
  • Extend VLA models from arm-centric manipulation (π-series) to humanoid whole-body control (GR00T).
A humanoid VLA using SONIC as controller
Week 4 Pico 4 hardware switch

New VR device for teleoperation

Purchase done
  • Meta Quest covered hands & head, but has no official leg motion tracker.
  • Switched to Pico 4 + body trackers so whole-body teleoperation has full-pose input.
Pico 4 VR setup
Pico 4
Week 5–6 Option A IK upper · RL lower

Option A — IK upper body + RL lower body

Hardware risk
Method
Upper bodyIK-based joint-position control
Lower bodypretrained RL locomotion policy
  • Accurate hand tracking, but unnatural motion — the lower body struggled to maintain balance during manipulation.
  • Robot damage occurred during data collection runs.
IK + RL demonstration
Decouple whole-body control
Hardware damage
Hardware damage during teleoperation
Decision Not suitable as the main data-collection method.
Week 7–8 Option B SONIC deployment

Option B — One policy for whole-body

Current main direction
Method
PolicySONIC · pretrained whole-body motion tracking
InputVR device motion / VLA command
Outputcoordinated full-body G1 motion
SONIC deployment · run 1
SONIC deployment · run 2
SONIC deployment · run 3
Dimension
IK + RL Policy
SONIC
Hand accuracy
High
Medium
Whole-body naturalness
Low
High
Balance during manipulation
Weak
Better
Suitability for data collection
Risky
Promising
Decision Use SONIC as the current main whole-body teleoperation direction for data collection.
Week 9 Collecting first-person demos

First-person VLA data collection

Data collecting
  • Target data: human teleoperation command + first-person visual observations.
  • Goal scope: ~300 trajectories to fine-tune a locomotion + manipulation policy.
Third-person · session view
First-person · onboard camera
Week 10 Camera OAK switch

New camera for humanoid VLA

Purchase done
  • Onboard RealSense 435 has a narrow field of view.
  • Switched to the OAK camera, used by recent egocentric humanoid VLA datasets such as EgoScale.
EgoScale dataset · OAK-camera egocentric capture
Week 11 Lower locked upper-body data only

Lock lower body, collect data

Hardware risk
  • Humanoid keeps warning right ankle overheating after ~5 minutes of walking.
  • Workaround: lock the lower body and continue collecting upper-body manipulation data.
  • New right humanoid leg has shipped from Hong Kong — ETA ~3 weeks.
Upper-body-only data collection · lower body suspended
Week 12 Psi0 VLA policy deployed

Psi0 VLA policy works

Done
  • Fine-tuned and deployed Psi0 VLA for upper-body manipulation on G1.
  • Fully autonomous, no teleoperation — the policy drives the arms directly from on-board observations.
Psi0 policy rollout · autonomous, no teleoperation
Milestone First end-to-end VLA policy running on the G1 from collected demonstrations — validates the data-collection pipeline from Weeks 1–11.
Summary Summary timeline takeaway

Timeline summary

Takeaway
  1. SONIC is the main teleoperation direction — IK + RL was tested first but proved unstable and damaged the robot.
  2. Hardware reliability is the main constraint — a right-ankle thermal issue currently limits collection to upper-body data.
  3. Psi0 VLA policy deployed — first autonomous rollout on G1 from collected demonstrations, validating the full pipeline.
End · Week 12