chore: release v0.2.0
This commit is contained in:
motphys-developers
167
docs/source/en/user_guide/demo/bring_ball.md
Normal file
167
docs/source/en/user_guide/demo/bring_ball.md
Normal file
@@ -0,0 +1,167 @@
|
||||
# Manipulator Bring Ball
|
||||
|
||||
Bring Ball is a classic Manipulator task from the DeepMind Control Suite. A planar hand with a thumb and finger must grasp a ball and move it to a target ball position. MotrixLab currently provides one Bring Ball environment:
|
||||
|
||||
- `dm-manipulator-bring-ball`: grasp the ball and move it to the target position
|
||||
|
||||
```{video} /_static/videos/bring-ball.mp4
|
||||
:poster: _static/images/poster/bring-ball.jpg
|
||||
:nocontrols:
|
||||
:autoplay:
|
||||
:playsinline:
|
||||
:muted:
|
||||
:loop:
|
||||
:width: 100%
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task Description
|
||||
|
||||
Bring Ball is a planar (x-z) grasp-and-transport task:
|
||||
|
||||
- The hand has 4 arm joints (`arm_root`, `arm_shoulder`, `arm_elbow`, `arm_wrist`) plus thumb/finger joints
|
||||
- Grasping is driven by the `grasp` tendon, coupling the `thumb` and `finger` joints
|
||||
- The ball can slide in the plane (`ball_x`, `ball_z`) and rotate about the y-axis (`ball_y`)
|
||||
- The target ball is a mocap body, sampled at reset
|
||||
|
||||
---
|
||||
|
||||
## Action Space
|
||||
|
||||
| Item | Details |
|
||||
| ------------- | ------------------------------- |
|
||||
| **Type** | `Box(-1.0, 1.0, (5,), float32)` |
|
||||
| **Dimension** | 5 |
|
||||
|
||||
Actions correspond to the following actuators:
|
||||
|
||||
| Index | Action Meaning | Min Control | Max Control | XML Name |
|
||||
| ----: | ---------------------------------- | :---------: | :---------: | :--------: |
|
||||
| 0 | Root joint drive | -1 | 1 | `root` |
|
||||
| 1 | Shoulder joint drive | -1 | 1 | `shoulder` |
|
||||
| 2 | Elbow joint drive | -1 | 1 | `elbow` |
|
||||
| 3 | Wrist joint drive | -1 | 1 | `wrist` |
|
||||
| 4 | Grasp drive (thumb/finger coupled) | -1 | 1 | `grasp` |
|
||||
|
||||
---
|
||||
|
||||
## Observation Space
|
||||
|
||||
| Item | Details |
|
||||
| ------------- | -------------------------------- |
|
||||
| **Type** | `Box(-inf, inf, (41,), float32)` |
|
||||
| **Dimension** | 41 |
|
||||
|
||||
The observation vector is composed of the following parts (in order):
|
||||
|
||||
| Part | Content Description | Dim | Notes |
|
||||
| -------------- | ----------------------------- | --- | ------------------------------------ |
|
||||
| **arm_pos** | `sin`/`cos` of 8 joint angles | 16 | Joint order listed below |
|
||||
| **arm_vel** | 8 joint velocities | 8 | Same order as arm_pos |
|
||||
| **touch** | `log(1 + touch)` sensors | 5 | palm/finger/thumb/fingertip/thumbtip |
|
||||
| **hand_pos** | Grasp site world position | 3 | x, y, z |
|
||||
| **object_pos** | Ball position | 3 | x, y, z |
|
||||
| **target_pos** | Target ball position | 3 | x, y, z |
|
||||
| **rel** | `object_pos - target_pos` | 3 | Relative position |
|
||||
|
||||
| Index | Observation Range | Dim | Notes |
|
||||
| ----- | ----------------------------------------------- | --- | --------------------------------- |
|
||||
| 0-15 | `sin`/`cos` of 8 joint angles | 16 | Joint order: arm_root -> thumbtip |
|
||||
| 16-23 | 8 joint velocities | 8 | Same order as above |
|
||||
| 24-28 | Touch: palm, finger, thumb, fingertip, thumbtip | 5 | `log(1 + touch)` |
|
||||
| 29-31 | Grasp position (x, y, z) | 3 | hand_pos |
|
||||
| 32-34 | Ball position (x, y, z) | 3 | object_pos |
|
||||
| 35-37 | Target ball position (x, y, z) | 3 | target_pos |
|
||||
| 38-40 | Ball relative position (x, y, z) | 3 | rel |
|
||||
|
||||
Joint order: `arm_root`, `arm_shoulder`, `arm_elbow`, `arm_wrist`, `finger`, `fingertip`, `thumb`, `thumbtip`.
|
||||
|
||||
---
|
||||
|
||||
## Reward Function Design
|
||||
|
||||
Bring Ball uses shaped rewards with multiple components and penalties:
|
||||
|
||||
```python
|
||||
# R1: Reach - fingertips approach the ball
|
||||
r_reach = tolerance(avg_tip_dist)
|
||||
|
||||
# R2: Orient - palm points toward the ball
|
||||
r_orient = clip(1 - orient_bound + dot(hand_dir, unit_vec_to_ball), 0..1)
|
||||
|
||||
# R3: Pause - reduce arm jitter when close to the ball
|
||||
r_pause = tolerance(arm_speed_step) * is_close_to_ball
|
||||
|
||||
# R4: Close - grasp intent and contact together
|
||||
r_close = r_close_intent * (approach_or_grasp)
|
||||
|
||||
# R5: Lift & Transport - height and target distance
|
||||
r_lift_height = tolerance(ball_z)
|
||||
r_transport = tolerance(move_dist_to_target)
|
||||
r_lift = mix(r_lift_height, r_transport)
|
||||
|
||||
# Precision/Progress
|
||||
r_precision = tolerance(move_dist_to_target, gaussian)
|
||||
r_progress = (prev_dist - curr_dist) * scale
|
||||
|
||||
# Penalties
|
||||
penalty_side + penalty_hover
|
||||
```
|
||||
|
||||
Default weights (from `BringBallCfg`):
|
||||
|
||||
- reach 1.0, orient 1.5, pause 0.5, close 2.0, lift 6.0, precision 1.0
|
||||
- lift mixes `lift_height_weight` and `transport_weight`
|
||||
- progress reward is controlled by `transport_progress_scale`
|
||||
|
||||
---
|
||||
|
||||
## Initial State
|
||||
|
||||
- **Arm initialization**: uses the default model pose (`randomize_arm=False`), thumb/finger are symmetric
|
||||
- **Target position**: `x in [-0.4, 0.4]`, `z in [0.1, 0.4]`, `y = 0.001`
|
||||
- **Ball position**: `x in [-0.4, 0.4]`, `z in [0.2, 0.7]`, with a minimum hand distance
|
||||
- **Physics settling**: performs settle steps after reset (`settle_steps=300`)
|
||||
|
||||
---
|
||||
|
||||
## Episode Termination Conditions
|
||||
|
||||
- Terminate if any `NaN` appears in observations
|
||||
|
||||
---
|
||||
|
||||
## Usage Guide
|
||||
|
||||
### 1. Environment Preview (random actions)
|
||||
|
||||
```bash
|
||||
uv run scripts/view.py --env dm-manipulator-bring-ball
|
||||
```
|
||||
|
||||
### 2. Start Training
|
||||
|
||||
```bash
|
||||
uv run scripts/train.py --env dm-manipulator-bring-ball --train-backend torch
|
||||
```
|
||||
|
||||
### 3. View Training Progress
|
||||
|
||||
```bash
|
||||
uv run tensorboard --logdir runs/dm-manipulator-bring-ball
|
||||
```
|
||||
|
||||
### 4. Test Training Results
|
||||
|
||||
```bash
|
||||
uv run scripts/play.py --env dm-manipulator-bring-ball
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Training Results
|
||||
|
||||
1. The hand consistently reaches and grasps the ball
|
||||
2. The ball is lifted off the ground and held at a stable height
|
||||
3. The ball is reliably moved close to the target ball position
|
||||
Reference in New Issue
Block a user