cyy/go2_rl_gym
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

v1.1.1-rc2; add README

Browse Source
This commit is contained in:
wty-yy
2026-01-27 21:36:44 +08:00
parent 064d0c90ea
commit 93c815b2d9
10 changed files with 468 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
README.md
View File

@@ -1,152 +1,172 @@
<div align="center">
<h1 align="center">Go2 RL GYM</h1>
<p align="center">
<a href="README_en.md">🌎 English</a> | <span>🇨🇳 中文</span>
</p>
<h1 align="center">Go2 RL GYM</h1>
<p align="center">
<span>🌎 English</span> | <a href="README_zh.md">🇨🇳 中文</a>
</p>
</div>
<p align="center">
<strong>本仓库基于<a href="https://github.com/unitreerobotics/unitree_rl_gym">unitree_rl_gym</a>，使用强化学习训练Go2机器狗。</strong>
<strong>This repository builds on <a href="https://github.com/unitreerobotics/unitree_rl_gym">unitree_rl_gym</a> to train the Unitree Go2 quadruped with reinforcement learning.</strong>
</p>
<div align="center">
| <div align="center"> Isaac Gym </div> | <div align="center"> Mujoco </div> | <div align="center"> Physical </div> |
| <div align="center"> Isaac Gym </div> | <div align="center"> Mujoco </div> | <div align="center"> Physical </div> |
|--- | --- | --- |
| TODO | TODO | TODO |
| ![isaacgym eval](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/isaacgym_eval.gif) | ![mujoco eval](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/mujoco_eval.gif) | ![real eval](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/real_eval.gif) |
</div>
## 📦 安装配置
## 📦 Installation
安装和配置步骤请参考 [setup.md](/doc/setup_zh.md)
Follow the step-by-step setup guide in [setup.md](doc/setup_en.md).
## 🛠️ 使用指南
## 🛠️ Usage Guide
### 1. 训练
### 1. Train
运行以下命令进行训练：
Run the following command to launch training:
```bash
python legged_gym/scripts/train.py --task=xxx
```
#### ⚙️ 参数说明
- `--task`: 必选参数，值可选(go2, go2_cts, go2_moe_cts)
- `--headless`: 默认启动图形界面，设为 true 时不渲染图形界面（效率更高）
- `--resume`: 从日志中选择 checkpoint 继续训练
- `--experiment_name`: 运行/加载的 experiment 名称
- `--run_name`: 运行/加载的 run 名称
- `--load_run`: 加载运行的名称，默认加载最后一次运行
- `--checkpoint`: checkpoint 编号，默认加载最新一次文件
- `--num_envs`: 并行训练的环境个数
- `--seed`: 随机种子
- `--max_iterations`: 训练的最大迭代次数
- `--sim_device`: 仿真计算设备，指定 CPU 为 `--sim_device=cpu`
- `--rl_device`: 强化学习计算设备，指定 CPU 为 `--rl_device=cpu`
#### ⚙️ Arguments
- `--task`: Required. Options include `go2`, `go2_cts`, `go2_moe_cts`, `go2_moe_ng_cts`, `go2_mcp_cts`, `go2_ac_moe_cts`, `go2_dual_moe_cts`; `go2_moe_cts` is the paper's final version.
- `--headless`: Render viewer by default; set to `true` to disable rendering for higher throughput.
- `--resume`: Resume training from a chosen checkpoint in the logs.
- `--experiment_name`: Experiment folder to save/load from.
- `--run_name`: Run subfolder name to save/load from.
- `--load_run`: Name of the run to load (defaults to the most recent run).
- `--checkpoint`: Checkpoint index to load (defaults to the latest file).
- `--num_envs`: Number of parallel simulated environments.
- `--seed`: Random seed.
- `--max_iterations`: Maximum training iterations.
- `--sim_device`: Physics simulation device. Use `--sim_device=cpu` to force CPU.
- `--rl_device`: RL computation device. Use `--rl_device=cpu` to force CPU.
- `--robogauge`: Enable RoboGauge evaluation tool; disabled by default. Evaluation results are saved as `results_{it}.yaml` in `logs/{exp_name}/{date}/robogauge_results` and logged to TensorBoard.
- `--robogauge_port`: RoboGauge server port; default is 9973.
**默认保存训练结果**：`logs/<experiment_name>/<date_time>_<run_name>/model_<iteration>.pt`
> RoboGauge evaluation requires a separate server to be started. Refer to the [RoboGauge documentation](https://github.com/wty-yy/RoboGauge).
**Default checkpoint path**: `logs/<experiment_name>/<date_time>_<run_name>/model_<iteration>.pt`
---
### 2. Play
如果想要在 Gym 中查看训练效果，可以运行以下命令：
Visualize policies inside Gym with:
```bash
python legged_gym/scripts/play.py --task=xxx
```
**说明**：
**Notes**
- Play 启动参数为随机地形，难度在7到9之间。
- 默认加载实验文件夹最新训练的一个模型。
- 可通过 `experiment_name` 和 `checkpoint` 指定其他模型，例如
```bash
python legged_gym/scripts/play.py --task=go2_cts --num_envs 100 --experiment_name go2_cts_hard_terrain --checkpoint 100000
```
- Play launches on randomized terrain with difficulty between 7 and 9.
- It automatically loads the latest checkpoint inside the experiment folder.
- Override via `experiment_name` and `checkpoint`, for example:
```bash
python legged_gym/scripts/play.py --task=go2_cts --num_envs 100 --experiment_name go2_cts_hard_terrain --checkpoint 100000
```
#### 💾 导出网络
#### 💾 Policy Export
Play 会导出 Actor 网络，保存于 `logs/{experiment_name}/exported/policies` 中：
- `policy.pt`: torch script模型，用于Sim2Sim。
- `policy.onnx`: onnx模型，用于Sim2Real。
- `policy.pkl`: 模型权重。
Play exports the Actor network to `logs/{experiment_name}/exported/policies`:
- `policy.pt`: TorchScript model for Sim2Sim.
- `policy.onnx`: ONNX model for Sim2Real.
- `policy.pkl`: Raw weights.
#### Play 效果
#### Demonstration
| Go2 |
|--- |
| TODO |
![isaacgym play](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/isaacgym_play.gif)
---
### 3. Sim2Sim (Mujoco)
支持在 Mujoco 仿真器中运行 Sim2Sim：
Run policies in the Mujoco simulator:
```bash
python deploy/deploy_mujoco/deploy_go2.py
```
如果有xbox协议的手柄接入主机，自动切换为手柄控制，否则只会保持默认指令前进。
Connect an Xbox-compatible gamepad to enable teleoperation; otherwise, the agent keeps a default forward command.
- 替换网络模型：默认模型位于 `deploy/pre_train/go2/go2_cts_150k.pt`；自己训练模型保存于`logs/{experiment_name}/exported/policies/policy.pt`，只需替换 yaml 配置文件中 `policy_path`。
- 替换环境地形：默认地形为 `resources/robots/go2/stairs.xml`，其他可选地形，平地 `flat.xml`，赛道 `race_track.xml`，地形使用[terrain_generator.py](resources/robots/go2/terrain_generator.py)生成，参考[unitree_mujoco/terrain_tool](https://github.com/unitreerobotics/unitree_mujoco/tree/main/terrain_tool)。
- **Swap the policy**: The default checkpoint is `deploy/pre_train/go2/go2_cts_150k.pt`. Replace `policy_path` in the YAML config with your own `logs/{experiment_name}/exported/policies/policy.pt`.
- **Swap terrains**: Default terrain is `resources/robots/go2/stairs.xml`. Alternatives include `flat.xml` and `race_track.xml`. Generate new terrains with [terrain_generator.py](resources/robots/go2/terrain_generator.py) (see also [unitree_mujoco/terrain_tool](https://github.com/unitreerobotics/unitree_mujoco/tree/main/terrain_tool)).
#### 运行效果
#### Results
| 平地 | 台阶 | 赛道 |
| Flat | Stairs | Race Track |
|--- | --- | --- |
| | | |
| ![eval flat](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/mujoco_eval_flat.gif) | ![eval stairs](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/mujoco_eval.gif) | ![eval track](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/go2_rl_gym/mujoco_eval_track.gif) |
---
### 4. Sim2Real
#### 4.1 Python实物部署 （需要安装 unitree_sdk2_python）
#### 4.1 Python Deployment (requires [unitree_sdk2_python](https://github.com/unitreerobotics/unitree_sdk2_python))
先用app进入设备→服务状态→点击运控服务，关闭`mcf`，打开`ota_box`服务。
```bash
# Onboard Jetson: pick Python by JetPack version
# JetPack 6: Python 3.10
# JetPack 5: Python 3.8
conda create -n deploy python=3.10
conda activate deploy
# Install the matching PyTorch wheel for your Jetson
# https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048
git clone https://github.com/unitreerobotics/unitree_sdk2_python.git
cd unitree_sdk2_python
pip3 install -e .
```
In the Unitree app, open Device → Service, disable `mcf/*`, and enable the `ota_box` service.
Assuming the interface to the low-level controller is `eth0`:
假设和下位机连接的网卡名称为`eth0`，执行
```bash
cd deploy/deploy_real
python deploy_real_go2.py eth0
```
`start`站立，`A`启动控制
#### 4.2 C++实物部署（需要安装 unitree_cpp_deploy）
Press `start` to stand and `A` to engage the controller.
参考[unitree_cpp_deploy](https://github.com/wty-yy-mini/unitree_cpp_deploy)使用说明。
#### 4.2 C++ Deployment (requires unitree_cpp_deploy)
#### 运行效果
Follow the usage described in [unitree_cpp_deploy](https://github.com/wty-yy-mini/unitree_cpp_deploy).
| Python部署 | C++部署 |
#### Demonstration
| Python Deploy | C++ Deploy |
| --- | --- |
| | |
| ![python deploy](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/deploy/py_deploy_with_commands.gif) | ![cpp deploy](https://raw.githubusercontent.com/robogauge/picture-bed/refs/heads/main/deploy/cpp_deploy_with_commands.gif) |
---
## 🎉 致谢
## 🎉 Acknowledgements
本仓库开发离不开以下开源项目的支持与贡献，特此感谢：
This repository would not exist without the following open-source projects:
- [unitree_rl_gym](https://github.com/unitreerobotics/unitree_rl_gym)：宇树机器人强化学习训练基础框架。
- [legged\_gym](https://github.com/leggedrobotics/legged_gym)：构建基础训练环境。
- [rsl\_rl](https://github.com/leggedrobotics/rsl_rl.git)：强化学习算法实现。
- [mujoco](https://github.com/google-deepmind/mujoco.git)：提供强大CPU仿真功能。
- [unitree\_sdk2\_python](https://github.com/unitreerobotics/unitree_sdk2_python.git)：实物部署硬件Python通信接口。
- [unitree_sdk2](https://github.com/unitreerobotics/unitree_sdk2)：实物部署硬件C++通信接口。
- [unitree_rl_gym](https://github.com/unitreerobotics/unitree_rl_gym): Unitree's core RL training framework.
- [legged_gym](https://github.com/leggedrobotics/legged_gym): Base locomotion environment.
- [rsl_rl](https://github.com/leggedrobotics/rsl_rl.git): Reinforcement learning algorithms.
- [mujoco](https://github.com/google-deepmind/mujoco.git): High-performance CPU physics simulator.
- [unitree_sdk2_python](https://github.com/unitreerobotics/unitree_sdk2_python.git): Python hardware interface for deployment.
- [unitree_sdk2](https://github.com/unitreerobotics/unitree_sdk2): C++ hardware interface for deployment.
本仓库实现包含以下论文，特此感谢：
Related publications implemented in this repo:
- [CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion](https://arxiv.org/pdf/2405.10830)
Contributors:
- [@windigal](https://github.com/windigal): CTS algorithm reproduction, video editing
- [@wertyuilife2](https://github.com/wertyuilife2): CTS algorithm reproduction
---
## 🔖 许可证
## 🔖 License
新增内容根据 [MIT License](./LICENSE) 授权，原仓库unitree_rl_gym根据 [BSD 3-Clause License](./LICENSE) 授权。
详情请阅读完整 [LICENSE 文件](./LICENSE)。
New contributions follow the [MIT License](LICENSE); the original unitree_rl_gym remains under the [BSD 3-Clause License](LICENSE).
See the complete [LICENSE file](LICENSE) for details.