chore: release v0.0.1

docs/source/en/index.md

@@ -0,0 +1,66 @@
+# MotrixLab
+
+MotrixLab is a general-purpose machine learning architecture designed for robot training. Built on the MotrixSim simulation platform, it enables policy training on both CPU and GPU, providing flexible and easy-to-use training capabilities for users across various operating systems and hardware devices.
+
+::::{grid} 1 2 3 3
+:gutter: 2 2 2 2
+
+:::{grid-item-card}
+
+```{video} _static/videos/cartpole.mp4
+:poster: _static/images/poster/cartpole.jpg
+:nocontrols:
+:autoplay:
+:playsinline:
+:muted:
+:loop:
+:width: 100%
+```
+
+:::
+
+:::{grid-item-card}
+
+```{video} _static/videos/go1_walk.mp4
+:poster: _static/images/poster/go1_walk.jpg
+:nocontrols:
+:autoplay:
+:playsinline:
+:muted:
+:loop:
+:width: 100%
+```
+
+:::
+
+:::{grid-item-card}
+
+```{video} _static/videos/dm_walker.mp4
+:poster: _static/images/poster/dm_walker.jpg
+:nocontrols:
+:autoplay:
+:playsinline:
+:muted:
+:loop:
+:width: 100%
+```
+
+:::
+::::
+
+## Key Features
+
+-   **Cross-Platform Software Support**: Compatible with Windows and Linux operating systems
+-   **CPU Simulation Support**: Supports CPU-based simulation, significantly reducing GPU requirements
+
+## Use Cases
+
+-   Robot control algorithm development and testing
+-   Building reinforcement learning environments
+-   Education and research
+
+```{toctree}
+:maxdepth: 1
+
+user_guide/index
+```

docs/source/en/user_guide/demo/cartpole.md

@@ -0,0 +1,112 @@
+# CartPole Training Example
+
+CartPole is a classic control task in reinforcement learning. The goal is to keep the pole balanced by controlling the cart's left-right movement.
+![cartpole](/_static/images/poster/cartpole.jpg)
+
+## Task Description
+
+-   **State Space**: Cart position, cart velocity, pole angle, pole angular velocity
+-   **Action Space**: Apply force left or right
+-   **Reward Function**: +1 reward for each step the pole stays upright
+-   **Termination Conditions**: Pole angle exceeds ±15 degrees or episode length exceeds 10 seconds
+
+## Quick Start
+
+### 1. Environment Preview
+
+```bash
+uv run scripts/view.py --env cartpole
+```
+
+### 2. Start Training
+
+```bash
+# Train with default parameters
+uv run scripts/train.py --env cartpole
+
+# Customize number of environments
+uv run scripts/train.py --env cartpole --num-envs 1024
+
+# Enable rendering (visualize during training)
+uv run scripts/train.py --env cartpole --render
+```
+
+### 3. View Training Progress
+
+```bash
+uv run tensorboard --logdir runs/cartpole
+```
+
+### 4. Test Training Results
+
+```bash
+# Automatically find best policy for testing (recommended)
+uv run scripts/play.py --env cartpole
+
+# Manually specify policy file for testing
+uv run scripts/play.py --env cartpole --policy runs/cartpole/nn/best_policy.pickle
+```
+
+> **Tip**: The system will automatically find the latest and best policy files in the `runs/cartpole/` directory for testing. You can also manually specify specific policy files using the `--policy` parameter.
+
+## Configuration Parameters
+
+Main configuration parameters for the CartPole environment:
+
+```python
+@dataclass
+class CartPoleEnvCfg(EnvCfg):
+    model_file: str = "path/to/inverted_pendulum.xml"  # MJCF model file
+    reset_noise_scale: float = 0.01                    # Reset noise
+    max_episode_seconds: float = 10.0                 # Maximum episode length
+```
+
+Training configuration parameters:
+
+```python
+from dataclasses import dataclass
+from motrix_rl.skrl.cfg import PPOCfg
+from motrix_rl import registry
+
+@registry.rlcfg("cartpole")
+@dataclass
+class CartPolePPO(PPOCfg):
+    max_env_steps: int = 10_000_000          # Maximum environment steps
+    check_point_interval: int = 500          # Checkpoint interval
+
+    # Network structure (small network suitable for simple tasks)
+    policy_hidden_layer_sizes: tuple[int, ...] = (32, 32)
+    value_hidden_layer_sizes: tuple[int, ...] = (32, 32)
+
+    # PPO parameters
+    rollouts: int = 32                       # Experience replay rounds
+    learning_epochs: int = 5                 # Training rounds
+    mini_batches: int = 4                    # Number of mini-batches
+```
+
+**Note**: CartPole is a simple task and currently uses universal configuration. If you need to create specialized configurations for different training backends (JAX/Torch), refer to the environment configuration documentation examples.
+
+## Custom Training
+
+You can override default configurations through command line arguments:
+
+```bash
+uv run scripts/train.py --env cartpole \
+  --num-envs 1024 \
+  --train-backend jax \
+  --sim-backend np
+```
+
+## Expected Results
+
+-   Pole angle stays within ±5 degrees most of the time
+-   Cart displacement range is reasonable
+
+## Troubleshooting
+
+If training performance is poor, you can try:
+
+1. Adjust learning rate (try 1e-4 to 1e-3)
+2. Increase number of environments (more parallel training)
+3. Adjust reward function weights
+4. Check if physical parameters are reasonable

docs/source/en/user_guide/demo/dm_walker.md

@@ -0,0 +1,153 @@
+# 2D Walker Robot Training Example
+
+The 2D Walker Robot (Walker2D) is a classic robot control task from DeepMind Control Suite. The goal is to achieve standing, walking, and running by controlling the robot's joints.
+
+```{video} /_static/videos/dm_walker.mp4
+:poster: _static/images/poster/dm_walker.jpg
+:nocontrols:
+:autoplay:
+:playsinline:
+:muted:
+:loop:
+:width: 100%
+```
+
+## Task Description
+
+Walker2D is a 2D planar bipedal robot with multiple joints and actuators:
+
+-   **State Space**: Includes rotation angles and angular velocities of various robot parts, torso height and velocity, etc.
+-   **Action Space**: Control torques for each joint
+-   **Reward Function**: Mainly composed of maintaining standing balance and forward speed
+-   **Termination Conditions**: Robot falls or joints reach limit positions
+
+### Three Task Modes
+
+1. **dm-stander**: Static standing task (move_speed = 0.0)
+
+```bash
+uv run scripts/train.py --env dm-stander
+```
+
+2. **dm-walker**: Walking task (move_speed = 1.0)
+
+```bash
+uv run scripts/train.py --env dm-walker
+```
+
+3. **dm-runner**: Running task (move_speed = 5.0)
+
+```bash
+uv run scripts/train.py --env dm-runner
+```
+
+## Quick Start
+
+### 1. Environment Preview
+
+```bash
+# View standing task
+uv run scripts/view.py --env dm-stander
+
+# View walking task
+uv run scripts/view.py --env dm-walker
+
+# View running task
+uv run scripts/view.py --env dm-runner
+```
+
+### 2. Start Training
+
+```bash
+# Train standing task
+uv run scripts/train.py --env dm-stander
+
+# Train walking task (default)
+uv run scripts/train.py --env dm-walker
+
+# Train running task
+uv run scripts/train.py --env dm-runner
+
+# Customize number of environments
+uv run scripts/train.py --env dm-walker --num-envs 512
+
+# Enable rendering (visualize during training)
+uv run scripts/train.py --env dm-walker --render
+```
+
+### 3. View Training Progress
+
+```bash
+uv run tensorboard --logdir runs/dm-walker
+```
+
+### 4. Test Training Results
+
+```bash
+# Automatically find best policy for testing (recommended)
+uv run scripts/play.py --env dm-walker
+
+# Manually specify policy file for testing
+uv run scripts/play.py --env dm-walker --policy runs/dm-walker/nn/best_policy.pickle
+```
+
+> **Tip**: The system will automatically find the latest and best policy files in the `runs/dm-walker/` directory for testing. Supports dm-stander, dm-walker, dm-runner three task modes.
+
+## Configuration Parameters
+
+### Environment Configuration
+
+```python
+@dataclass
+class WalkerEnvCfg(EnvCfg):
+    model_file: str = "walker.xml"           # MJCF model file
+    max_episode_seconds: float = 25.0        # Maximum episode length
+    sim_dt: float = 0.0125                   # Simulation time step
+    ctrl_dt: float = 0.025                   # Control time step
+    move_speed: float = 1.0                  # Target movement speed
+    stand_height: float = 1.2                # Target standing height
+```
+
+### Training Configuration
+
+```python
+@dataclass
+class WalkerRLCfg(BaseRLCfg):
+    num_envs: int = 512                      # Number of parallel environments
+    learning_rate: float = 3e-4              # Learning rate
+    batch_size: int = 512                    # Batch size
+    max_epochs: int = 1000                   # Maximum training epochs
+```
+
+## Reward Function Design
+
+Walker2D's reward function consists of the following components:
+
+### Basic Standing Reward
+
+```python
+# Height reward: keep torso at target height
+
+# Upright reward: keep torso upright
+```
+
+### Movement Reward (walking and running tasks)
+
+```python
+# Speed reward: track target speed
+
+# Total reward = standing reward * movement weight
+```
+
+## Expected Results
+
+1. **dm-stander**:
+
+    - Torso height maintained in 1.0-1.4m range
+
+2. **dm-walker**:
+
+    - Actual walking speed close to 1.0 m/s
+
+3. **dm-runner**:
+    - Running speed reaches 4.0-5.0 m/s

docs/source/en/user_guide/demo/locomotion_unitree_go1.md

@@ -0,0 +1,169 @@
+# Unitree GO1 Robot Walking Training Example
+
+Unitree GO1 is a quadruped robot platform. This example demonstrates how to train GO1 to achieve stable gait walking on flat terrain.
+
+```{video} /_static/videos/go1_walk.mp4
+:poster: _static/images/poster/go1_walk.jpg
+:nocontrols:
+:autoplay:
+:playsinline:
+:muted:
+:loop:
+:width: 100%
+```
+
+## Task Description
+
+The GO1 quadruped robot has 12 degrees of freedom (3 joints per leg) and needs to learn coordinated gait control through deep reinforcement learning:
+
+-   **State Space**: 48-dimensional, including robot linear velocity, angular velocity, posture, joint angles, joint velocities, actions, and commands
+-   **Action Space**: 12-dimensional, controlling target positions of each joint (converted to torques through PD controller)
+-   **Reward Function**: Composite reward including speed tracking, posture stability, energy efficiency, and other components
+-   **Termination Conditions**: Robot trunk contacts ground or other unstable states
+
+### Training Task
+
+```bash
+uv run scripts/train.py --env go1-flat-terrain-walk
+```
+
+## Configuration Parameters
+
+### Environment Configuration
+
+```python
+@dataclass
+class Go1WalkNpEnvCfg(EnvCfg):
+    max_episode_seconds: float = 20.0      # Maximum episode length
+    model_file: str = "scene_motor_actuator.xml"
+    sim_dt: float = 0.01                   # Simulation time step
+    ctrl_dt: float = 0.01                  # Control time step
+```
+
+### Training Configuration
+
+```python
+from dataclasses import dataclass
+from motrix_rl.skrl.cfg import PPOCfg
+from motrix_rl import registry
+
+@registry.rlcfg("go1-flat-terrain-walk")
+@dataclass
+class Go1WalkPPO(PPOCfg):
+    """
+    GO1 quadruped robot walking training configuration
+    """
+
+    seed = 42
+    max_env_steps: int = 40960000          # Maximum training steps
+    num_envs: int = 2048                   # Number of parallel environments
+
+    # Large network structure (suitable for complex robot control tasks)
+    policy_hidden_layer_sizes: tuple[int, ...] = (512, 256, 128)
+    value_hidden_layer_sizes: tuple[int, ...] = (512, 256, 128)
+
+    # PPO parameters (optimized for robot tasks)
+    learning_epochs: int = 2               # Training rounds
+    mini_batches: int = 32                 # Number of mini-batches
+    learning_rate: float = 1e-3            # Learning rate
+```
+
+**Note**: GO1 is a complex task that uses large network structures. If you need to create specialized configurations for different training backends (JAX/Torch), refer to the environment configuration documentation examples.
+
+### Control Configuration
+
+```python
+@dataclass
+class ControlConfig:
+    stiffness = 80                         # PD controller stiffness [N*m/rad]
+    damping = 1                            # PD controller damping [N*m*s/rad]
+    action_scale = 0.1                     # Action scaling factor
+```
+
+### Initial Joint Angles
+
+```python
+default_joint_angles = {
+    "FL_hip": 0.0,      # Front left hip joint
+    "RL_hip": 0.0,      # Rear left hip joint
+    "FR_hip": -0.0,     # Front right hip joint
+    "RR_hip": -0.0,     # Rear right hip joint
+    "FL_thigh": 0.9,    # Front left thigh
+    "RL_thigh": 0.9,    # Rear left thigh
+    "FR_thigh": 0.9,    # Front right thigh
+    "RR_thigh": 0.9,    # Rear right thigh
+    "FL_calf": -1.8,    # Front left calf
+    "RL_calf": -1.8,    # Rear left calf
+    "FR_calf": -1.8,    # Front right calf
+    "RR_calf": -1.8,    # Rear right calf
+}
+```
+
+## Reward Function Design
+
+GO1's reward function is a complex composite function containing multiple components:
+
+### Main Reward Components
+
+```python
+reward_config.scales = {
+    "tracking_lin_vel": 1.0,      # Linear velocity tracking reward
+    "tracking_ang_vel": 0.5,      # Angular velocity tracking reward
+    "feet_air_time": 1.0,         # Foot air time reward
+    "lin_vel_z": -2.0,            # Z-axis linear velocity penalty
+    "ang_vel_xy": -0.05,          # XY-axis angular velocity penalty
+    "orientation": -0.0,          # Posture deviation penalty
+    "torques": -0.00001,          # Torque consumption penalty
+    "dof_acc": -2.5e-7,           # Joint acceleration penalty
+    "action_rate": -0.001,        # Action change rate penalty
+    "hip_pos": -1,                # Hip joint position penalty
+    "calf_pos": -0.3,             # Calf joint position penalty
+}
+```
+
+### Key Reward Functions
+
+#### Velocity Tracking Reward
+
+```python
+# Track linear velocity commands (xy plane)
+def _reward_tracking_lin_vel(self, data, commands):
+
+# Track angular velocity commands (yaw)
+def _reward_tracking_ang_vel(self, data, commands):
+```
+
+#### Foot Air Time Reward
+
+```python
+def _reward_feet_air_time(self, commands, info):
+```
+
+## Observation Space Composition
+
+GO1's observation space is 48-dimensional, containing the following information:
+
+```python
+obs = np.hstack([
+    noisy_linvel,        # 3D: Local coordinate system linear velocity
+    noisy_gyro,          # 3D: Gyroscope data
+    local_gravity,       # 3D: Local gravity direction
+    noisy_joint_angle,   # 12D: Joint angles (relative to default values)
+    noisy_joint_vel,     # 12D: Joint velocities
+    last_actions,        # 12D: Previous frame actions
+    command,             # 3D: Velocity commands [vx, vy, vyaw]
+])
+```
+
+## Motion Velocity Command Generation
+
+Random velocity commands are generated during training to ensure the agent can track different movement speeds:
+
+```python
+def resample_commands(self, num_envs: int):
+```
+
+## Expected Training Results
+
+1. Stable quadruped gait
+2. Good speed tracking

docs/source/en/user_guide/getting_started/hello_motrixlab.md

@@ -0,0 +1,84 @@
+# Quick Start: Hello MotrixLab
+
+This tutorial demonstrates the MotrixLab workflow through a simple example - loading and training a cartpole environment:
+
+## Environment Preview
+
+We provide a simple script to visualize an environment without executing any training. This helps you verify that system dependencies are correctly configured:
+
+```bash
+uv run scripts/view.py --env cartpole
+```
+
+This will open a visualization window showing the cartpole physics simulation environment with random actions for demonstration.
+
+## Train Model
+
+Start training the cartpole balancing task:
+
+```bash
+uv run scripts/train.py --env cartpole
+```
+
+The training process will automatically:
+
+1. Automatically select training backend (JAX or PyTorch) based on hardware environment
+2. Create training environments
+3. Start PPO algorithm training
+
+Training results will be saved in the `runs/cartpole/` directory, including:
+
+-   Training checkpoints
+-   TensorBoard log files
+
+## Visualize Training Process
+
+If you want to observe the model's learning process during training, you can enable visualization rendering:
+
+```bash
+uv run scripts/train.py --env cartpole --render
+```
+
+### 🎮 Interactive Rendering Control
+
+> **Important Note**: Visualization significantly reduces training speed and is recommended mainly for debugging and demonstration purposes.
+
+During visualized training, you can use the **spacebar** to dynamically control rendering:
+
+-   **Enable Rendering**: Press spacebar to enable visualization and observe robot behavior
+-   **Disable Rendering**: Press spacebar again to disable rendering and improve training speed
+-   **Switch Anytime**: No need to restart the program; you can switch at any time during training
+
+This interactive control allows you to observe training effects when needed and enjoy fast training when not needed. This feature also works during inference.
+
+## View Training Results
+
+Use TensorBoard to view training progress:
+
+```bash
+uv run tensorboard --logdir runs/cartpole
+```
+
+## Test Trained Model
+
+After training is complete, test the trained policy:
+
+```bash
+# Automatically find best policy for testing (recommended)
+uv run scripts/play.py --env cartpole
+
+# Manually specify policy file for testing (if you need a specific version)
+uv run scripts/play.py --env cartpole --policy runs/cartpole/YOUR_RESULT_NUMBER/best_agent.pickle
+```
+
+> **Tip**: The system will automatically find the latest and best policy files in the `runs/cartpole/` directory. Usually, using the auto-discovery feature is sufficient.
+
+## That Completes Our Example
+
+Next, you can try modifying parameters to observe physical effects under different settings, or try other environments.
+
+## Next Steps
+
+-   Learn about the [Basic Framework](../tutorial/basic_frame.md)
+-   Study [Physics Environment Configuration](../tutorial/physics_environment.md)
+-   View more [Training Examples](../demo/cartpole.md)

docs/source/en/user_guide/getting_started/installation.md

@@ -0,0 +1,55 @@
+# Installation Environment
+
+## Installation Requirements
+
+-   **Python Version**: {bdg-danger-line}`3.10.*`
+
+    | Python Version | Support Status |
+    | :------------: | :------------: |
+    |     ≤ 3.9      |       ❌       |
+    |      3.10      |       ✅       |
+    |     ≥ 3.11     |       ❌       |
+
+-   **Package Manager**: {bdg-danger-line}`UV`
+
+-   **System and Architecture**:
+
+    -   {bdg-danger-line}`Windows(x86_64)`
+    -   {bdg-danger-line}`Linux(x86_64)`
+
+    ```{note}
+    Features supported on each platform:
+
+    | Operating System | CPU Simulation | Interactive Viewer | GPU Simulation |
+    | :--------------: | :------------: | :----------------: | :------------: |
+    |      Linux       |       ✅       |         ✅          |    🛠️ In Development    |
+    |     Windows      |       ✅       |         ✅          |    🛠️ In Development    |
+    ```
+
+## Installation Method
+
+### Clone Project
+
+```bash
+git clone https://github.com/Motphys/MotrixLab.git
+cd MotrixLab
+```
+
+### Install Dependencies
+
+Use UV to install project dependencies:
+
+```bash
+# Install all dependencies
+uv sync --all-packages --all-extras
+```
+
+If you only need to install one training backend, you can choose to install a specific backend type:
+
+```bash
+# Install SKRL JAX (support Linux only)
+uv sync --all-packages --extra skrl-jax
+
+# Install SKRL PyTorch
+uv sync --all-packages --extra skrl-torch
+```

docs/source/en/user_guide/index.md

@@ -0,0 +1,32 @@
+# User Guide
+
+```{toctree}
+:caption: Getting Started
+:maxdepth: 1
+
+getting_started/installation
+getting_started/hello_motrixlab
+
+```
+
+```{toctree}
+:caption: Tutorials
+:maxdepth: 1
+
+tutorial/basic_frame
+tutorial/physics_environment
+tutorial/training_environment_config
+tutorial/rewards
+tutorial/training_and_result
+
+```
+
+```{toctree}
+:caption: Training Examples
+:maxdepth: 1
+
+demo/cartpole
+demo/dm_walker
+demo/locomotion_unitree_go1
+
+```

docs/source/en/user_guide/tutorial/basic_frame.md

@@ -0,0 +1,149 @@
+# Basic Framework
+
+MotrixLab is a robot reinforcement learning platform. This section introduces MotrixLab's framework design and the relationships between various components. If you are already familiar with reinforcement learning content, you can skip directly to the next section to learn how to develop your own training environments.
+
+## MotrixLab's Framework Design
+
+MotrixLab adopts a layered architecture design, clearly separating training environments from training logic:
+
+```
+MotrixLab/
+├── motrix_envs/               # Environment layer: Physics simulation and task definition
+│   ├── basic/                  # Basic environments (cartpole, walker, etc.)
+│   ├── locomotion/             # Locomotion environments (GO1 robot, etc.)
+│   ├── np/                     # NumPy simulation backend framework
+│   ├── base.py                 # Environment base class
+│   └── registry.py             # Environment registry system
+├── motrix_rl/                # Training layer: RL algorithms and configuration
+│   ├── skrl/                   # SKRL framework integration (JAX/PyTorch)
+│   ├── base.py                 # RL configuration base class
+│   └── registry.py             # RL configuration registry system
+└── scripts
+    ├── train.py                # Training entry script
+    ├── play.py                 # Testing entry script
+    └── view.py                 # Visualization script
+```
+
+## Core Component Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                        User Interface Layer                      │
+│                    train.py │ play.py │ view.py                   │
+└─────────────────────────────────────────────────────────────────┘
+                                │
+                                ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                    Training Algorithm Layer (SKRL)              │
+│                 PPO Trainer │ Network Architecture │ Optimizer   │
+└─────────────────────────────────────────────────────────────────┘
+                                │
+                                ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                      Environment Implementation Layer            │
+│   Environment Config(EnvCfg) │ Environment Impl(Env) │ Reward   │
+└─────────────────────────────────────────────────────────────────┘
+                                │
+                                ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                    Physics Simulation Layer (MotrixSim)          │
+│                    MJCF Model │ Physics Engine │ Collision      │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Detailed Core Components
+
+### 1. Training Environment
+
+**Location**: Environment Implementation Layer
+
+The training environment is the core component of MotrixLab, containing three key parts:
+
+-   **Environment Configuration (EnvCfg)**: Defines physics simulation parameters (model files, time steps, episode length, etc.) and task-specific parameters
+-   **Environment Implementation (Env)**: Inherits from base environment class, implements specific task logic, physics simulation interaction, and termination condition checking
+-   **Reward Function (Reward)**: Implemented in the environment's step method, calculates reward values based on current state and actions
+
+Environments are registered to the system through decorators.
+
+### 2. Reward Function
+
+**Location**: Configuration Management Layer + Environment Implementation Layer
+
+The reward function in MotrixLab adopts a dual-structure design:
+
+-   **Configuration Level**: Define reward weights, reward component types, and scaling parameters in configuration classes
+-   **Implementation Level**: Calculate specific reward values in the environment's `_compute_reward` method based on configuration parameters
+
+This design allows reward functions to be flexibly adjusted through configuration files while implementing complex computational logic in code.
+
+### 3. Configuration Parameters
+
+**Location**: Configuration Management Layer
+
+Configuration parameters adopt a hierarchical management structure:
+
+-   **Environment Configuration (EnvCfg)**: Controls physics simulation and task behavior, including simulation parameters, reset noise, time limits, etc.
+-   **Training Configuration (RLCfg)**: Controls reinforcement learning algorithms, including network structure, learning rate, batch size, training steps, etc.
+
+Configuration classes support inheritance, parameter validation, and runtime overriding, ensuring parameter reasonableness and flexibility.
+
+### 4. Registry System
+
+**Location**: Hub connecting various components
+
+The registry system implements automatic component registration through the decorator pattern:
+
+-   Environment configuration classes are registered through `@registry.envcfg()`
+-   Environment implementation classes are registered through `@registry.env()`, supporting multiple backends
+-   RL configuration classes are registered through `@registry.rlcfg()`
+
+The registry system achieves component decoupling, making it simple and fast to add new environments or modify configurations.
+
+## Data Flow and Workflow
+
+### Training Process Overview
+
+```
+User Command → Configuration Parsing → Environment Creation → Training Loop → Model Save
+   ↓
+train.py --env cartpole
+   ↓
+Find Configuration Classes → Create Environment → Start PPO Training → Save Model
+```
+
+### Core Workflow
+
+1. **Environment Definition**: Create environment configuration classes and implementation classes in `src/motrix_envs/`
+2. **Automatic Registration**: Register components to the system through decorators
+3. **Configuration Loading**: When starting from command line, the system automatically finds and loads corresponding configurations
+4. **Environment Creation**: Factory pattern creates environment instances, supporting parameter override
+5. **Training Execution**: PPO algorithm interacts with the environment, collects data and updates policy
+6. **Result Saving**: Periodically save checkpoints and final models
+
+### Role of Configuration Parameters
+
+Configuration parameters play a key connecting role throughout the process:
+
+-   **Environment Configuration** determines physics simulation behavior (time steps, model files, noise, etc.)
+-   **Reward Configuration** affects learning signals (reward weights, calculation methods, etc.)
+-   **Training Configuration** controls algorithm behavior (network structure, learning rate, batch size, etc.)
+
+## Multi-Backend Support
+
+MotrixLab's layered design naturally supports multiple backends:
+
+-   **Simulation Backends**: MotrixSim (CPU)
+-   **Training Backends**: JAX and PyTorch, supporting GPU acceleration
+-   **Algorithm Framework**: Mainly integrates SKRL, easy to extend to other algorithms
+
+## Design Advantages
+
+This architecture design brings the following core advantages:
+
+1. **Module Decoupling**: Environment development and training logic are completely separated
+2. **Flexible Configuration**: Supports hierarchical configuration and runtime parameter override
+3. **Strong Extensibility**: Easily add new components through the registry system
+4. **Multi-Backend Compatibility**: Same environment can use different simulation and training backends
+5. **Experiment-Friendly**: Configurations can be saved and compared, ensuring experimental reproducibility
+
+Through this framework design, MotrixLab provides a clear, flexible, and easy-to-use development platform for robot reinforcement learning.

docs/source/en/user_guide/tutorial/physics_environment.md

@@ -0,0 +1,66 @@
+# Physics Environment Configuration
+
+Physics environment configuration defines simulation parameters and model file settings in reinforcement learning training.
+MotrixLab uses [MotrixSim](https://motrixsim.readthedocs.io/en/latest/user_guide/index.html) as the physics simulation backend.
+
+## Supported File Formats
+
+-   [**MJCF**](https://mujoco.readthedocs.io/en/stable/XMLreference.html) (MuJoCo XML format) - Provides rich physics features and simulation configuration
+
+## Model File Configuration
+
+You need to specify model file paths in environment configuration classes:
+
+```python
+
+@registry.envcfg("my-task")
+@dataclass
+class MyTaskEnvCfg(EnvCfg):
+    # Model file path (required)
+    model_file: str = "my_model.xml"
+
+    # Simulation time parameters
+    sim_dt: float = 0.002      # Simulation time step
+    ctrl_dt: float = 0.02      # Control update frequency
+
+    # Episode parameters
+    max_episode_seconds: float = 20.0
+    reset_noise_scale: float = 0.01
+```
+
+### Recommended Directory Structure
+
+```
+motrix_envs/my_task/
+├── __init__.py          # Module initialization
+├── cfg.py               # Environment configuration
+├── my_model.xml         # Physics model file
+└── my_env.py            # Environment implementation
+```
+
+For complex models with many referenced files, it's recommended to use folder management.
+
+## Common Configuration Issues
+
+### File Path Issues
+
+-   When using relative paths, ensure paths are relative to the configuration file location
+-   Avoid using hardcoded absolute paths
+-   Check file permissions and accessibility
+-   Ensure all referenced sub-files exist
+
+### Time Step Settings
+
+-   `ctrl_dt` should be an integer multiple of `sim_dt`
+-   `sim_dt` that is too small will affect simulation performance
+-   `ctrl_dt` that is too large will affect control precision
+-   Recommend `sim_dt` between 0.001-0.02 seconds
+
+### Simulation Stability
+
+-   Avoid excessively large time steps
+-   Set contact parameters reasonably to avoid penetration
+-   Mass and inertia distribution should be reasonable
+-   Joint limits should match actual conditions
+
+Through proper physics environment configuration, you can create accurate and efficient simulation environments for reinforcement learning training.

docs/source/en/user_guide/tutorial/rewards.md

@@ -0,0 +1,50 @@
+# Reward Function Design
+
+The reward function tells the agent what behaviors are desired and is a core part of reinforcement learning environment design.
+
+## Position of Reward Function in Training Loop
+
+In MotrixLab's NpEnv, reward calculation occurs in the `update_state` phase of the `step` function:
+
+```python
+# Execution flow of NpEnv.step()
+def step(self, actions: np.ndarray) -> NpEnvState:
+    # 1. Preparation phase: Clear rewards and state
+    self._prev_physics_step()  # reward = 0.0, terminated = False, truncated = False
+
+    # 2. Apply actions
+    self._state = self.apply_action(actions, self._state)
+
+    # 3. Physics simulation
+    self.physics_step()  # Execute physics simulation
+
+    # 4. Update state ← Reward function is calculated here
+    self._state = self.update_state(self._state)  # Calculate rewards and observations
+
+    # 5. Post-processing
+    self._update_truncate()  # Check time truncation
+    self._reset_done_envs()  # Reset completed environments
+
+    return self._state
+```
+
+You need to implement reward calculation logic in the `update_state` method of subclasses. For specific reward function design ideas, please refer to the training examples.
+
+### Reward Component Design Principles
+
+1. **Separation of Concerns**: Each reward function should handle a specific goal
+2. **Weight Configuration**: Manage weights of different components through configuration files
+3. **Normalization**: Keep reward values within reasonable ranges
+4. **Smoothness**: Avoid hard thresholds, use exponential functions for smooth transitions
+
+This approach makes reward functions modular, facilitating debugging and adjustment of individual component weights.
+
+## Design Principles
+
+1. **Clear Goal Orientation**: Reward functions should directly reflect task goals
+2. **Reasonable Reward Range**: Avoid overly large or small reward values to maintain training stability
+3. **Balance Exploration and Exploitation**: Appropriately reward behaviors close to goals, avoiding sparse rewards
+4. **Avoid Reward Hacking**: Check if agents can obtain high rewards through unintended means
+5. **Debug-Friendly**: Output reward decomposition information during development for optimization
+
+By correctly implementing reward calculation in the `update_state` method, you can design effective learning signals for various robot tasks.

docs/source/en/user_guide/tutorial/training_and_result.md

@@ -0,0 +1,92 @@
+# Training Execution and Result Analysis
+
+This section introduces how to execute reinforcement learning training and how to analyze and use training results.
+
+## Start Training
+
+### Basic Training Commands
+
+```bash
+# Train with default parameters
+uv run scripts/train.py --env cartpole
+
+# Specify simulation backend
+uv run scripts/train.py --env cartpole --sim-backend np
+
+# Specify training backend
+uv run scripts/train.py --env cartpole --train-backend jax
+uv run scripts/train.py --env cartpole --train-backend torch
+```
+
+### Advanced Training Configuration
+
+```bash
+# Customize training parameters
+uv run scripts/train.py --env cartpole \
+  --num-envs 1024 \
+  --train-backend jax \
+  --sim-backend np
+
+# Enable rendering to monitor training process
+uv run scripts/train.py --env cartpole --render
+```
+
+### Different Backend Configuration Differences
+
+The system supports configuring different reinforcement learning parameters for different training backends (JAX/Torch). For example:
+
+-   **dm-walker environment**:
+
+    -   JAX backend: `mini_batches: 4`
+    -   Torch backend: `mini_batches: 32`
+
+-   **dm-runner environment**:
+    -   JAX backend: `learning_epochs: 4`
+    -   Torch backend: `learning_epochs: 2`
+
+These differences are implemented through the `@rlcfg(env_name, backend="jax/torch")` decorator in configuration classes. The system automatically applies the corresponding configuration based on the selected training backend.
+
+### Supported Command Line Parameters
+
+| Parameter         | Description                     | Default Value |
+| ----------------- | ------------------------------- | ------------- |
+| `--env`           | Environment name                | `cartpole`    |
+| `--sim-backend`   | Simulation backend (np)         | Auto select   |
+| `--train-backend` | Training backend (jax/torch)    | Auto select   |
+| `--num-envs`      | Number of parallel environments | 2048          |
+| `--render`        | Enable rendering                | False         |
+
+> **Note**: Other parameters such as learning rate, network structure, etc., can be set in configuration files. Some environments support configuring different parameters for different training backends.
+
+## Training Process Monitoring
+
+### TensorBoard Monitoring
+
+Start TensorBoard to view training progress:
+
+```bash
+uv run tensorboard --logdir runs/{env-name}
+```
+
+For example:
+
+```bash
+uv run tensorboard --logdir runs/cartpole
+```
+
+## Model Evaluation and Testing
+
+### Using Trained Policies
+
+```bash
+# Automatically find best policy for testing (recommended)
+uv run scripts/play.py --env cartpole
+
+# Manually specify policy file for testing
+uv run scripts/play.py --env cartpole --policy runs/cartpole/nn/best_policy.pickle
+
+# Specify number of test environments
+uv run scripts/play.py --env cartpole --num-envs 100
+```
+
+> **Note**: The system will automatically find the latest and best policy files in the `runs/cartpole/` directory for testing.

docs/source/en/user_guide/tutorial/training_environment_config.md

@@ -0,0 +1,231 @@
+# Training Environment Configuration
+
+MotrixLab provides a flexible configuration system that allows users to customize reinforcement learning training parameters. This section introduces how to configure training environments and reinforcement learning algorithm parameters.
+
+## RL Training Configuration (PPOCfg)
+
+Training configuration defines parameters for reinforcement learning algorithms. MotrixLab now supports configuring different parameters for different training backends.
+
+### Basic Training Configuration
+
+```python
+from dataclasses import dataclass
+from motrix_rl.skrl.cfg import PPOCfg
+from motrix_rl import registry
+
+# Universal configuration (applies to all backends)
+@registry.rlcfg("my-task")
+@dataclass
+class MyTaskRLCfg(PPOCfg):
+    # Environment parameters
+    num_envs: int = 2048              # Number of parallel environments during training
+    play_num_envs: int = 16           # Number of parallel environments during evaluation
+
+    # PPO algorithm parameters
+    learning_rate: float = 3e-4       # Learning rate
+    rollouts: int = 32                # Experience replay rounds
+    learning_epochs: int = 10         # Number of epochs per update
+    mini_batches: int = 32            # Number of mini-batches
+    discount_factor: float = 0.99     # Discount factor
+    grad_norm_clip: float = 1.0       # Gradient clipping
+
+    # Network structure parameters
+    policy_hidden_layer_sizes: tuple[int, ...] = (256, 128, 64)  # Policy network hidden layers
+    value_hidden_layer_sizes: tuple[int, ...] = (256, 128, 64)   # Value network hidden layers
+
+    # Training control parameters
+    max_env_steps: int = 1_000_000    # Maximum environment steps
+    check_point_interval: int = 10_000 # Checkpoint interval
+```
+
+### Different Backend Configurations
+
+```python
+# JAX backend specific configuration
+@registry.rlcfg("my-task", backend="jax")
+@dataclass
+class MyTaskJAXCfg(PPOCfg):
+    # JAX optimized configuration
+    learning_rate: float = 2e-4       # JAX backend uses smaller learning rate
+    mini_batches: int = 4             # JAX supports large batches, fewer mini-batches
+    learning_epochs: int = 4          # Training rounds
+    num_envs: int = 2048              # More parallel environments
+
+# Torch backend specific configuration
+@registry.rlcfg("my-task", backend="torch")
+@dataclass
+class MyTaskTorchCfg(PPOCfg):
+    # Torch optimized configuration
+    learning_rate: float = 1e-4       # Torch backend uses even smaller learning rate
+    mini_batches: int = 32            # Torch needs more mini-batches
+    learning_epochs: int = 2          # Fewer training rounds
+    num_envs: int = 1024              # Fewer parallel environments
+```
+
+### Complete Configuration Example
+
+```python
+@dataclass
+class CompletePPOConfig(PPOCfg):
+    """
+    Complete reinforcement learning training configuration example
+    Contains all configuration parameters from basic to advanced
+    """
+
+    # ===== Basic Training Parameters =====
+    seed: Optional[int] = None         # Random seed
+    num_envs: int = 2048               # Number of parallel environments during training
+    play_num_envs: int = 16            # Number of parallel environments during evaluation
+    max_env_steps: int = 2_048_000     # Maximum training steps
+    check_point_interval: int = 1000   # Checkpoint save interval
+
+    # ===== PPO Algorithm Core Parameters =====
+    learning_rate: float = 3e-4        # Learning rate
+    rollouts: int = 32                 # Experience replay rounds
+    learning_epochs: int = 2           # Number of training rounds per update
+    mini_batches: int = 32             # Number of mini-batches
+    discount_factor: float = 0.99      # Discount factor
+    lambda_param: float = 0.95         # GAE parameter
+    grad_norm_clip: float = 1.0        # Gradient clipping
+
+    # ===== PPO Clipping Parameters =====
+    ratio_clip: float = 0.2            # PPO clipping ratio
+    value_clip: float = 0.2            # Value clipping
+    clip_predicted_values: bool = True # Clip predicted values
+
+    # ===== Loss Function Parameters =====
+    entropy_loss_scale: float = 0.0    # Entropy loss coefficient
+    value_loss_scale: float = 2.0      # Value loss coefficient
+    kl_threshold: float = 0            # KL divergence threshold
+
+    # ===== Learning Rate Scheduler =====
+    learning_rate_scheduler_kl_threshold: float = 0.008  # Adaptive learning rate KL threshold
+
+    # ===== Network Architecture Configuration =====
+    # Small network (suitable for simple tasks like CartPole)
+    # policy_hidden_layer_sizes: tuple[int, ...] = (128, 64)
+    # value_hidden_layer_sizes: tuple[int, ...] = (128, 64)
+
+    # Medium network (default configuration, suitable for most tasks)
+    policy_hidden_layer_sizes: tuple[int, ...] = (256, 128, 64)
+    value_hidden_layer_sizes: tuple[int, ...] = (256, 128, 64)
+
+    # Large network (suitable for complex tasks like robot control)
+    # policy_hidden_layer_sizes: tuple[int, ...] = (512, 256, 128)
+    # value_hidden_layer_sizes: tuple[int, ...] = (512, 256, 128)
+
+    # ===== Network Sharing Configuration =====
+    share_policy_value_features: bool = True  # Policy and value networks share feature extraction layers
+
+    # ===== Training Control Parameters =====
+    random_timesteps: int = 0          # Random timesteps
+    learning_starts: int = 0           # Timesteps to start learning
+    time_limit_bootstrap: bool = True  # Time limit bootstrap
+
+    # ===== Reward Shaping =====
+    rewards_shaper_scale: float = 1.0  # Reward scaling factor
+```
+
+## Configuration Usage Methods
+
+### 1. Default Configuration Usage
+
+```bash
+# Use configuration given in code
+uv run scripts/train.py --env my-task
+
+# Specify training backend, system will automatically select corresponding backend configuration
+uv run scripts/train.py --env my-task --train-backend jax
+uv run scripts/train.py --env my-task --train-backend torch
+```
+
+### 2. Command Line Parameter Override
+
+```bash
+# Override supported command line parameters
+uv run scripts/train.py --env my-task \
+  --num-envs 1024 \
+  --train-backend jax \
+  --sim-backend np
+
+# System will automatically select JAX backend configuration
+```
+
+### 3. Configuration Priority
+
+System selects configuration in the following priority:
+
+1. **Backend-specific configuration**: If there is a configuration decorated with `@rlcfg(env_name, backend="jax/torch")`
+2. **Universal configuration**: If there is a configuration decorated with `@rlcfg(env_name)` (no backend parameter)
+3. **Configuration override**: Command line `cfg_override` parameter
+
+For example:
+
+```python
+# Highest priority - backend specific configuration
+@rlcfg("my-task", backend="jax")
+@dataclass
+class MyTaskJAXCfg(PPOCfg):
+    mini_batches: int = 4
+
+# Second priority - universal configuration
+@rlcfg("my-task")
+@dataclass
+class MyTaskRLCfg(PPOCfg):
+    mini_batches: int = 32
+
+# When using --train-backend jax, system will select MyTaskJAXCfg
+# When using --train-backend torch, system will select MyTaskRLCfg
+```
+
+#### User Configurable Parameters
+
+| MotrixLab Configuration Class          | SKRL Framework Parameter                      | Description                         |
+| -------------------------------------- | --------------------------------------------- | ----------------------------------- |
+| `learning_rate`                        | `learning_rate`                               | Learning rate                       |
+| `rollouts`                             | `rollouts`                                    | Experience replay rounds            |
+| `learning_epochs`                      | `learning_epochs`                             | Training rounds                     |
+| `mini_batches`                         | `mini_batches`                                | Number of mini-batches              |
+| `discount_factor`                      | `discount_factor`                             | Discount factor                     |
+| `grad_norm_clip`                       | `grad_norm_clip`                              | Gradient clipping                   |
+| `lambda_param`                         | `lambda`                                      | GAE parameter                       |
+| `ratio_clip`                           | `ratio_clip`                                  | PPO clipping ratio                  |
+| `value_clip`                           | `value_clip`                                  | Value clipping                      |
+| `clip_predicted_values`                | `clip_predicted_values`                       | Clip predicted values               |
+| `entropy_loss_scale`                   | `entropy_loss_scale`                          | Entropy loss coefficient            |
+| `value_loss_scale`                     | `value_loss_scale`                            | Value loss coefficient              |
+| `kl_threshold`                         | `kl_threshold`                                | KL divergence threshold             |
+| `random_timesteps`                     | `random_timesteps`                            | Random timesteps                    |
+| `learning_starts`                      | `learning_starts`                             | Learning start timesteps            |
+| `time_limit_bootstrap`                 | `time_limit_bootstrap`                        | Time limit bootstrap                |
+| `learning_rate_scheduler_kl_threshold` | `learning_rate_scheduler_kwargs.kl_threshold` | Adaptive learning rate KL threshold |
+| `check_point_interval`                 | `experiment.write_interval`                   | Log write interval                  |
+| `check_point_interval`                 | `experiment.checkpoint_interval`              | Checkpoint save interval            |
+| `rewards_shaper_scale`                 | `rewards_shaper`                              | Reward scaling function             |
+
+#### Preprocessor Parameters
+
+| SKRL Framework Parameter | Type                  | Description         |
+| ------------------------ | --------------------- | ------------------- |
+| `state_preprocessor`     | RunningStandardScaler | State normalization |
+| `value_preprocessor`     | RunningStandardScaler | Value normalization |
+
+### Configuration Hierarchy Summary
+
+```
+User Configuration Class (PPOCfg)
+    ↓ Backend specific selection
+Backend Configuration (JAX/Torch)
+    ↓ Parameter mapping
+SKRL Framework Configuration Dictionary
+    ↓ Pass to
+PPO Agent
+    ↓ Execute
+Reinforcement Learning Training
+```
+
+This design allows users to:
+
+1. Control complex training parameters through simple configuration classes
+2. Configure different parameters for different training backends to achieve optimal performance
+3. Maintain full compatibility with the SKRL framework