cyy/Motrixlab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: release v0.0.1

Browse Source
This commit is contained in:
motphys-developers
2025-11-20 08:57:48 +00:00
commit 5133830b5a
105 changed files with 8789 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

112
docs/source/en/user_guide/demo/cartpole.md Normal file
View File

@@ -0,0 +1,112 @@
# CartPole Training Example
CartPole is a classic control task in reinforcement learning. The goal is to keep the pole balanced by controlling the cart's left-right movement.
![cartpole](/_static/images/poster/cartpole.jpg)
## Task Description
- **State Space**: Cart position, cart velocity, pole angle, pole angular velocity
- **Action Space**: Apply force left or right
- **Reward Function**: +1 reward for each step the pole stays upright
- **Termination Conditions**: Pole angle exceeds ±15 degrees or episode length exceeds 10 seconds
## Quick Start
### 1. Environment Preview
```bash
uv run scripts/view.py --env cartpole
```
### 2. Start Training
```bash
# Train with default parameters
uv run scripts/train.py --env cartpole
# Customize number of environments
uv run scripts/train.py --env cartpole --num-envs 1024
# Enable rendering (visualize during training)
uv run scripts/train.py --env cartpole --render
```
### 3. View Training Progress
```bash
uv run tensorboard --logdir runs/cartpole
```
### 4. Test Training Results
```bash
# Automatically find best policy for testing (recommended)
uv run scripts/play.py --env cartpole
# Manually specify policy file for testing
uv run scripts/play.py --env cartpole --policy runs/cartpole/nn/best_policy.pickle
```
> **Tip**: The system will automatically find the latest and best policy files in the `runs/cartpole/` directory for testing. You can also manually specify specific policy files using the `--policy` parameter.
## Configuration Parameters
Main configuration parameters for the CartPole environment:
```python
@dataclass
class CartPoleEnvCfg(EnvCfg):
model_file: str = "path/to/inverted_pendulum.xml" # MJCF model file
reset_noise_scale: float = 0.01 # Reset noise
max_episode_seconds: float = 10.0 # Maximum episode length
```
Training configuration parameters:
```python
from dataclasses import dataclass
from motrix_rl.skrl.cfg import PPOCfg
from motrix_rl import registry
@registry.rlcfg("cartpole")
@dataclass
class CartPolePPO(PPOCfg):
max_env_steps: int = 10_000_000 # Maximum environment steps
check_point_interval: int = 500 # Checkpoint interval
# Network structure (small network suitable for simple tasks)
policy_hidden_layer_sizes: tuple[int, ...] = (32, 32)
value_hidden_layer_sizes: tuple[int, ...] = (32, 32)
# PPO parameters
rollouts: int = 32 # Experience replay rounds
learning_epochs: int = 5 # Training rounds
mini_batches: int = 4 # Number of mini-batches
```
**Note**: CartPole is a simple task and currently uses universal configuration. If you need to create specialized configurations for different training backends (JAX/Torch), refer to the environment configuration documentation examples.
## Custom Training
You can override default configurations through command line arguments:
```bash
uv run scripts/train.py --env cartpole \
--num-envs 1024 \
--train-backend jax \
--sim-backend np
```
## Expected Results
- Pole angle stays within ±5 degrees most of the time
- Cart displacement range is reasonable
## Troubleshooting
If training performance is poor, you can try:
1. Adjust learning rate (try 1e-4 to 1e-3)
2. Increase number of environments (more parallel training)
3. Adjust reward function weights
4. Check if physical parameters are reasonable

153
docs/source/en/user_guide/demo/dm_walker.md Normal file
View File

@@ -0,0 +1,153 @@
# 2D Walker Robot Training Example
The 2D Walker Robot (Walker2D) is a classic robot control task from DeepMind Control Suite. The goal is to achieve standing, walking, and running by controlling the robot's joints.
```{video} /_static/videos/dm_walker.mp4
:poster: _static/images/poster/dm_walker.jpg
:nocontrols:
:autoplay:
:playsinline:
:muted:
:loop:
:width: 100%
```
## Task Description
Walker2D is a 2D planar bipedal robot with multiple joints and actuators:
- **State Space**: Includes rotation angles and angular velocities of various robot parts, torso height and velocity, etc.
- **Action Space**: Control torques for each joint
- **Reward Function**: Mainly composed of maintaining standing balance and forward speed
- **Termination Conditions**: Robot falls or joints reach limit positions
### Three Task Modes
1. **dm-stander**: Static standing task (move_speed = 0.0)
```bash
uv run scripts/train.py --env dm-stander
```
2. **dm-walker**: Walking task (move_speed = 1.0)
```bash
uv run scripts/train.py --env dm-walker
```
3. **dm-runner**: Running task (move_speed = 5.0)
```bash
uv run scripts/train.py --env dm-runner
```
## Quick Start
### 1. Environment Preview
```bash
# View standing task
uv run scripts/view.py --env dm-stander
# View walking task
uv run scripts/view.py --env dm-walker
# View running task
uv run scripts/view.py --env dm-runner
```
### 2. Start Training
```bash
# Train standing task
uv run scripts/train.py --env dm-stander
# Train walking task (default)
uv run scripts/train.py --env dm-walker
# Train running task
uv run scripts/train.py --env dm-runner
# Customize number of environments
uv run scripts/train.py --env dm-walker --num-envs 512
# Enable rendering (visualize during training)
uv run scripts/train.py --env dm-walker --render
```
### 3. View Training Progress
```bash
uv run tensorboard --logdir runs/dm-walker
```
### 4. Test Training Results
```bash
# Automatically find best policy for testing (recommended)
uv run scripts/play.py --env dm-walker
# Manually specify policy file for testing
uv run scripts/play.py --env dm-walker --policy runs/dm-walker/nn/best_policy.pickle
```
> **Tip**: The system will automatically find the latest and best policy files in the `runs/dm-walker/` directory for testing. Supports dm-stander, dm-walker, dm-runner three task modes.
## Configuration Parameters
### Environment Configuration
```python
@dataclass
class WalkerEnvCfg(EnvCfg):
model_file: str = "walker.xml" # MJCF model file
max_episode_seconds: float = 25.0 # Maximum episode length
sim_dt: float = 0.0125 # Simulation time step
ctrl_dt: float = 0.025 # Control time step
move_speed: float = 1.0 # Target movement speed
stand_height: float = 1.2 # Target standing height
```
### Training Configuration
```python
@dataclass
class WalkerRLCfg(BaseRLCfg):
num_envs: int = 512 # Number of parallel environments
learning_rate: float = 3e-4 # Learning rate
batch_size: int = 512 # Batch size
max_epochs: int = 1000 # Maximum training epochs
```
## Reward Function Design
Walker2D's reward function consists of the following components:
### Basic Standing Reward
```python
# Height reward: keep torso at target height
# Upright reward: keep torso upright
```
### Movement Reward (walking and running tasks)
```python
# Speed reward: track target speed
# Total reward = standing reward * movement weight
```
## Expected Results
1. **dm-stander**:
- Torso height maintained in 1.0-1.4m range
2. **dm-walker**:
- Actual walking speed close to 1.0 m/s
3. **dm-runner**:
- Running speed reaches 4.0-5.0 m/s

169
docs/source/en/user_guide/demo/locomotion_unitree_go1.md Normal file
View File

@@ -0,0 +1,169 @@
# Unitree GO1 Robot Walking Training Example
Unitree GO1 is a quadruped robot platform. This example demonstrates how to train GO1 to achieve stable gait walking on flat terrain.
```{video} /_static/videos/go1_walk.mp4
:poster: _static/images/poster/go1_walk.jpg
:nocontrols:
:autoplay:
:playsinline:
:muted:
:loop:
:width: 100%
```
## Task Description
The GO1 quadruped robot has 12 degrees of freedom (3 joints per leg) and needs to learn coordinated gait control through deep reinforcement learning:
- **State Space**: 48-dimensional, including robot linear velocity, angular velocity, posture, joint angles, joint velocities, actions, and commands
- **Action Space**: 12-dimensional, controlling target positions of each joint (converted to torques through PD controller)
- **Reward Function**: Composite reward including speed tracking, posture stability, energy efficiency, and other components
- **Termination Conditions**: Robot trunk contacts ground or other unstable states
### Training Task
```bash
uv run scripts/train.py --env go1-flat-terrain-walk
```
## Configuration Parameters
### Environment Configuration
```python
@dataclass
class Go1WalkNpEnvCfg(EnvCfg):
max_episode_seconds: float = 20.0 # Maximum episode length
model_file: str = "scene_motor_actuator.xml"
sim_dt: float = 0.01 # Simulation time step
ctrl_dt: float = 0.01 # Control time step
```
### Training Configuration
```python
from dataclasses import dataclass
from motrix_rl.skrl.cfg import PPOCfg
from motrix_rl import registry
@registry.rlcfg("go1-flat-terrain-walk")
@dataclass
class Go1WalkPPO(PPOCfg):
"""
GO1 quadruped robot walking training configuration
"""
seed = 42
max_env_steps: int = 40960000 # Maximum training steps
num_envs: int = 2048 # Number of parallel environments
# Large network structure (suitable for complex robot control tasks)
policy_hidden_layer_sizes: tuple[int, ...] = (512, 256, 128)
value_hidden_layer_sizes: tuple[int, ...] = (512, 256, 128)
# PPO parameters (optimized for robot tasks)
learning_epochs: int = 2 # Training rounds
mini_batches: int = 32 # Number of mini-batches
learning_rate: float = 1e-3 # Learning rate
```
**Note**: GO1 is a complex task that uses large network structures. If you need to create specialized configurations for different training backends (JAX/Torch), refer to the environment configuration documentation examples.
### Control Configuration
```python
@dataclass
class ControlConfig:
stiffness = 80 # PD controller stiffness [N*m/rad]
damping = 1 # PD controller damping [N*m*s/rad]
action_scale = 0.1 # Action scaling factor
```
### Initial Joint Angles
```python
default_joint_angles = {
"FL_hip": 0.0, # Front left hip joint
"RL_hip": 0.0, # Rear left hip joint
"FR_hip": -0.0, # Front right hip joint
"RR_hip": -0.0, # Rear right hip joint
"FL_thigh": 0.9, # Front left thigh
"RL_thigh": 0.9, # Rear left thigh
"FR_thigh": 0.9, # Front right thigh
"RR_thigh": 0.9, # Rear right thigh
"FL_calf": -1.8, # Front left calf
"RL_calf": -1.8, # Rear left calf
"FR_calf": -1.8, # Front right calf
"RR_calf": -1.8, # Rear right calf
}
```
## Reward Function Design
GO1's reward function is a complex composite function containing multiple components:
### Main Reward Components
```python
reward_config.scales = {
"tracking_lin_vel": 1.0, # Linear velocity tracking reward
"tracking_ang_vel": 0.5, # Angular velocity tracking reward
"feet_air_time": 1.0, # Foot air time reward
"lin_vel_z": -2.0, # Z-axis linear velocity penalty
"ang_vel_xy": -0.05, # XY-axis angular velocity penalty
"orientation": -0.0, # Posture deviation penalty
"torques": -0.00001, # Torque consumption penalty
"dof_acc": -2.5e-7, # Joint acceleration penalty
"action_rate": -0.001, # Action change rate penalty
"hip_pos": -1, # Hip joint position penalty
"calf_pos": -0.3, # Calf joint position penalty
}
```
### Key Reward Functions
#### Velocity Tracking Reward
```python
# Track linear velocity commands (xy plane)
def _reward_tracking_lin_vel(self, data, commands):
# Track angular velocity commands (yaw)
def _reward_tracking_ang_vel(self, data, commands):
```
#### Foot Air Time Reward
```python
def _reward_feet_air_time(self, commands, info):
```
## Observation Space Composition
GO1's observation space is 48-dimensional, containing the following information:
```python
obs = np.hstack([
noisy_linvel, # 3D: Local coordinate system linear velocity
noisy_gyro, # 3D: Gyroscope data
local_gravity, # 3D: Local gravity direction
noisy_joint_angle, # 12D: Joint angles (relative to default values)
noisy_joint_vel, # 12D: Joint velocities
last_actions, # 12D: Previous frame actions
command, # 3D: Velocity commands [vx, vy, vyaw]
])
```
## Motion Velocity Command Generation
Random velocity commands are generated during training to ensure the agent can track different movement speeds:
```python
def resample_commands(self, num_envs: int):
```
## Expected Training Results
1. Stable quadruped gait
2. Good speed tracking