2D Walker Robot Training Example

The 2D Walker Robot (Walker2D) is a classic robot control task from DeepMind Control Suite. The goal is to achieve standing, walking, and running by controlling the robot's joints.

:poster: _static/images/poster/dm_walker.jpg
:nocontrols:
:autoplay:
:playsinline:
:muted:
:loop:
:width: 100%

Task Description

Walker2D is a 2D planar bipedal robot with multiple joints and actuators:

State Space: Includes rotation angles and angular velocities of various robot parts, torso height and velocity, etc.
Action Space: Control torques for each joint
Reward Function: Mainly composed of maintaining standing balance and forward speed
Termination Conditions: Robot falls or joints reach limit positions

Three Task Modes

dm-stander: Static standing task (move_speed = 0.0)

uv run scripts/train.py --env dm-stander

dm-walker: Walking task (move_speed = 1.0)

uv run scripts/train.py --env dm-walker

dm-runner: Running task (move_speed = 5.0)

uv run scripts/train.py --env dm-runner

Quick Start

1. Environment Preview

# View standing task
uv run scripts/view.py --env dm-stander

# View walking task
uv run scripts/view.py --env dm-walker

# View running task
uv run scripts/view.py --env dm-runner

2. Start Training

# Train standing task
uv run scripts/train.py --env dm-stander

# Train walking task (default)
uv run scripts/train.py --env dm-walker

# Train running task
uv run scripts/train.py --env dm-runner

# Customize number of environments
uv run scripts/train.py --env dm-walker --num-envs 512

# Enable rendering (visualize during training)
uv run scripts/train.py --env dm-walker --render

3. View Training Progress

uv run tensorboard --logdir runs/dm-walker

4. Test Training Results

# Automatically find best policy for testing (recommended)
uv run scripts/play.py --env dm-walker

# Manually specify policy file for testing
uv run scripts/play.py --env dm-walker --policy runs/dm-walker/nn/best_policy.pickle

Tip

: The system will automatically find the latest and best policy files in the runs/dm-walker/ directory for testing. Supports dm-stander, dm-walker, dm-runner three task modes.

Configuration Parameters

Environment Configuration

@dataclass
class WalkerEnvCfg(EnvCfg):
    model_file: str = "walker.xml"           # MJCF model file
    max_episode_seconds: float = 25.0        # Maximum episode length
    sim_dt: float = 0.0125                   # Simulation time step
    ctrl_dt: float = 0.025                   # Control time step
    move_speed: float = 1.0                  # Target movement speed
    stand_height: float = 1.2                # Target standing height

Training Configuration

@dataclass
class WalkerRLCfg(BaseRLCfg):
    num_envs: int = 512                      # Number of parallel environments
    learning_rate: float = 3e-4              # Learning rate
    batch_size: int = 512                    # Batch size
    max_epochs: int = 1000                   # Maximum training epochs

Reward Function Design

Walker2D's reward function consists of the following components:

Basic Standing Reward

# Height reward: keep torso at target height

# Upright reward: keep torso upright

Movement Reward (walking and running tasks)

# Speed reward: track target speed

# Total reward = standing reward * movement weight

Expected Results

dm-stander:
- Torso height maintained in 1.0-1.4m range
dm-walker:
- Actual walking speed close to 1.0 m/s
dm-runner:
- Running speed reaches 4.0-5.0 m/s

3.7 KiB Raw Blame History