go2_rl_gym/README.md

Go2 RL GYM

🌎 English | 🇨🇳 中文

This repository builds on unitree_rl_gym to train the Unitree Go2 quadruped with reinforcement learning.

Isaac Gym	Mujoco	Physical

📦 Installation

Follow the step-by-step setup guide in setup.md.

🛠️ Usage Guide

1. Train

Run the following command to launch training:

python legged_gym/scripts/train.py --task=xxx

⚙️ Arguments

• --task: Required. Options include go2, go2_cts, go2_moe_cts, go2_moe_ng_cts, go2_mcp_cts, go2_ac_moe_cts, go2_dual_moe_cts; go2_moe_cts is the paper's final version.
• --headless: Render viewer by default; set to true to disable rendering for higher throughput.
• --resume: Resume training from a chosen checkpoint in the logs.
• --experiment_name: Experiment folder to save/load from.
• --run_name: Run subfolder name to save/load from.
• --load_run: Name of the run to load (defaults to the most recent run).
• --checkpoint: Checkpoint index to load (defaults to the latest file).
• --num_envs: Number of parallel simulated environments.
• --seed: Random seed.
• --max_iterations: Maximum training iterations.
• --sim_device: Physics simulation device. Use --sim_device=cpu to force CPU.
• --rl_device: RL computation device. Use --rl_device=cpu to force CPU.
• --robogauge: Enable RoboGauge evaluation tool; disabled by default. Evaluation results are saved as results_{it}.yaml in logs/{exp_name}/{date}/robogauge_results and logged to TensorBoard.
• --robogauge_port: RoboGauge server port; default is 9973.

RoboGauge evaluation requires a separate server to be started. Refer to the RoboGauge documentation.

Default checkpoint path: logs/<experiment_name>/<date_time>_<run_name>/model_<iteration>.pt

---

2. Play

Visualize policies inside Gym with:

python legged_gym/scripts/play.py --task=xxx

Notes

• Play launches on randomized terrain with difficulty between 7 and 9.
• It automatically loads the latest checkpoint inside the experiment folder.
• Override via experiment_name and checkpoint, for example:

  python legged_gym/scripts/play.py --task=go2_cts --num_envs 100 --experiment_name go2_cts_hard_terrain --checkpoint 100000
  

💾 Policy Export

Play exports the Actor network to logs/{experiment_name}/exported/policies:

• policy.pt: TorchScript model for Sim2Sim.
• policy.onnx: ONNX model for Sim2Real.
• policy.pkl: Raw weights.

Demonstration

---

3. Sim2Sim (Mujoco)

Run policies in the Mujoco simulator:

python deploy/deploy_mujoco/deploy_go2.py

Connect an Xbox-compatible gamepad to enable teleoperation; otherwise, the agent keeps a default forward command.

• Swap the policy: The default checkpoint is deploy/pre_train/go2/go2_cts_150k.pt. Replace policy_path in the YAML config with your own logs/{experiment_name}/exported/policies/policy.pt.
• Swap terrains: Default terrain is resources/robots/go2/stairs.xml. Alternatives include flat.xml and race_track.xml. Generate new terrains with terrain_generator.py (see also unitree_mujoco/terrain_tool).

Results

Flat	Stairs	Race Track

---

4. Sim2Real

4.1 Python Deployment (requires unitree_sdk2_python)

# Onboard Jetson: pick Python by JetPack version
# JetPack 6: Python 3.10
# JetPack 5: Python 3.8
conda create -n deploy python=3.10
conda activate deploy
# Install the matching PyTorch wheel for your Jetson
# https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
cd unitree_sdk2_python
pip3 install -e .

In the Unitree app, open Device → Service, disable mcf/*, and enable the ota_box service.

Assuming the interface to the low-level controller is eth0:

cd deploy/deploy_real
python deploy_real_go2.py eth0

Press start to stand and A to engage the controller.

4.2 C++ Deployment (requires unitree_cpp_deploy)

Follow the usage described in unitree_cpp_deploy.

Demonstration

Python Deploy	C++ Deploy

---

🎉 Acknowledgements

This repository would not exist without the following open-source projects:

• unitree_rl_gym: Unitree's core RL training framework.
• legged_gym: Base locomotion environment.
• rsl_rl: Reinforcement learning algorithms.
• mujoco: High-performance CPU physics simulator.
• unitree_sdk2_python: Python hardware interface for deployment.
• unitree_sdk2: C++ hardware interface for deployment.

Related publications implemented in this repo:

• CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion

Contributors:

• @windigal: CTS algorithm reproduction, video editing
• @wertyuilife2: CTS algorithm reproduction

---

🔖 License

New contributions follow the MIT License; the original unitree_rl_gym remains under the BSD 3-Clause License.

See the complete LICENSE file for details.