58 lines
1.5 KiB
Markdown
58 lines
1.5 KiB
Markdown
# DreamWaQ 复刻进度
|
||
|
||
## 总览
|
||
|
||
| Phase | 状态 | 完成时间 |
|
||
|-------|------|----------|
|
||
| Phase 1: 环境差距补齐 | ✅ | 2026-06-30 |
|
||
| Phase 2: 地形生成 10×20 | ✅ | 2026-06-30 |
|
||
| Phase 3: CENet 网络集成 | ✅ | 2026-06-30 |
|
||
| Phase 4: 训练 Pipeline | ✅ | 2026-06-30 |
|
||
| Phase 5: 验证 | ⏳ | - |
|
||
|
||
## Phase 3+4: CENet + 训练
|
||
|
||
**新增文件**：
|
||
- `motrix_rl/src/motrix_rl/rslrl/torch/models/cenet_actor.py` — CENetVAE + CENetActorModel
|
||
- `motrix_rl/src/motrix_rl/rslrl/torch/train/dreamwaq_ppo.py` — DreamWaQPPO（PPO + VAE loss）
|
||
|
||
**修改文件**：
|
||
- `wrap_vec_env.py` — 多键 TensorDict（policy + obs_history + privileged_obs）
|
||
- `cfg.py` — 添加 vae_beta/cenet_in_dim/cenet_out_dim
|
||
- `go1.py` — go1-dreamwaq-walk 训练配置
|
||
|
||
**冒烟测试**：1 iteration 通过，VAE 损失 ≈ 2.85
|
||
|
||
### 数据流
|
||
|
||
```
|
||
DreamWaQTask.update_observation()
|
||
→ state.info["obs_history"] (N, 5, 45)
|
||
→ state.info["privileged_obs"] (N, 247)
|
||
|
||
RslrlNpEnvWrap._build_obs_dict()
|
||
→ TensorDict({
|
||
"policy": (N, 45),
|
||
"obs_history": (N, 225),
|
||
"privileged_obs": (N, 247),
|
||
})
|
||
|
||
CENetActorModel.get_latent()
|
||
→ VAE(obs_history) → code(19)
|
||
→ cat(code, policy) → latent(64)
|
||
→ MLP[512,256,128] → action(12)
|
||
|
||
DreamWaQPPO.update()
|
||
→ + VAE loss（速度估计 + 重建 + KL）
|
||
```
|
||
|
||
## 训练命令
|
||
|
||
```bash
|
||
# 平坦地形
|
||
uv run scripts/train.py --env go1-dreamwaq-walk --rllib rslrl
|
||
|
||
# 金字塔地形
|
||
DREAMWAQ_TERRAIN=pyramid uv run scripts/train.py --env go1-dreamwaq-walk --rllib rslrl
|
||
```
|