chore: release v0.1.0

(cherry picked from commit 82525f882f3924a332d9ce40bf64255d0d14f6a4)
2026-01-04 04:43:04 +00:00
parent 13cfbce9a7
commit 62011bb24f
390 changed files with 18897 additions and 626 deletions
--- a/docs/source/en/user_guide/demo/dm_reacher.md
+++ b/docs/source/en/user_guide/demo/dm_reacher.md
@@ -0,0 +1,128 @@
+# Two-Joint Robotic Arm Control
+
+Reacher is a classic robotic arm control task, simulating a robotic arm composed of two links. The goal is to bring the end effector (fingertip) as close as possible to a randomly generated target point.
+
+```{video} /_static/videos/dm_reacher.mp4
+:poster: _static/images/poster/dm_reacher.jpg
+:nocontrols:
+:autoplay:
+:playsinline:
+:muted:
+:loop:
+:width: 100%
+```
+
+## Task Description
+
+The Reacher consists of two joints, with two links connected by hinge joints. The objective of the task is to move the end of the robotic arm to the target position. The target point is randomly sampled at the beginning of each episode.
+
+---
+
+## Action Space
+
+| Item          | Details                         |
+| ------------- | ------------------------------- |
+| **Type**      | `Box(-1.0, 1.0, (2,), float32)` |
+| **Dimension** | 2                               |
+
+The actions correspond to:
+
+| Index | Action Description                               | Min Control | Max Control | XML Name | Joint Type |
+| ----- | ------------------------------------------------ | ----------- | ----------- | -------- | ---------- |
+| 0     | Torque applied to the first joint (root link)    | -1          | 1           | joint0   | hinge      |
+| 1     | Torque applied to the second joint (middle link) | -1          | 1           | joint1   | hinge      |
+
+---
+
+## Observation Space
+
+| Item          | Details                         |
+| ------------- | ------------------------------- |
+| **Type**      | `Box(-inf, inf, (6,), float32)` |
+| **Dimension** | 6                               |
+
+The observation vector contains the following parts (in order):
+
+-   **qpos**: 2 joint angles
+-   **fingertip → target vector difference**: x and y dimensions
+-   **qvel**: 2 joint angular velocities
+
+| Index | Observation                     | Min  | Max | XML Name   | Joint | Unit  |
+| ----- | ------------------------------- | ---- | --- | ---------- | ----- | ----- |
+| 0     | First joint angle               | -inf | inf | joint0_pos | hinge | rad   |
+| 1     | Second joint angle              | -inf | inf | joint1_pos | hinge | rad   |
+| 2     | fingertip - target x difference | -inf | inf | NA         | slide | m     |
+| 3     | fingertip - target y difference | -inf | inf | NA         | slide | m     |
+| 4     | First joint angular velocity    | -inf | inf | joint0_vel | hinge | rad/s |
+| 5     | Second joint angular velocity   | -inf | inf | joint1_vel | hinge | rad/s |
+
+---
+
+## Reward Function Design
+
+The reward for this task is based on the **distance between the fingertip and the target**:
+
+### Distance Reward (tolerance reward)
+
+```text
+reward = tolerance(|| fingertip - target ||)
+```
+
+-   The closer the distance, the higher the reward
+
+---
+
+## Initial State
+
+The initial state is sampled from random distributions:
+
+-   Arm angles: uniform distribution
+-   Arm angular velocities: small random values
+-   Target point position: random position in a circular area
+
+---
+
+## Episode Termination Conditions
+
+### Termination
+
+If `NaN` appears in the observations
+
+### Termination Handling
+
+```text
+reward = 0
+terminated = True
+```
+
+---
+
+## Usage Guide
+
+### 1. Environment Preview
+
+```bash
+uv run scripts/view.py --env dm-reacher
+```
+
+### 2. Start Training
+
+```bash
+uv run scripts/train.py --env dm-reacher
+```
+
+### 3. View Training Progress
+
+```bash
+uv run tensorboard --logdir runs/dm-reacher
+```
+
+### 4. Test Training Results
+
+```bash
+uv run scripts/play.py --env dm-reacher
+```
+
+## Expected Training Results
+
+The robotic arm quickly and accurately reaches the target point