chore: release v0.1.0
(cherry picked from commit 82525f882f3924a332d9ce40bf64255d0d14f6a4)
This commit is contained in:
motphys-developers
130
docs/source/en/user_guide/demo/bounce_ball.md
Normal file
130
docs/source/en/user_guide/demo/bounce_ball.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# Ping Pong Ball Bouncing
|
||||
|
||||
Train a single-arm robotic manipulator to control a paddle for continuous ball bouncing, maintaining the ball at a target height and position.
|
||||
|
||||
```{video} /_static/videos/bounce_ball.mp4
|
||||
:poster: _static/images/poster/bounce_ball.jpg
|
||||
:nocontrols:
|
||||
:autoplay:
|
||||
:playsinline:
|
||||
:muted:
|
||||
:loop:
|
||||
:width: 100%
|
||||
```
|
||||
|
||||
## Task Description
|
||||
|
||||
Bounce Ball is a single-arm robotic manipulation task using a 6-DOF Peitian AIR4-560 industrial robotic arm to control the position of an end-effector paddle. The agent controls the position changes of the arm's 6 joints as actions, making the ping pong ball bounce continuously on the paddle and keeping it as close as possible to the target height and target horizontal position.
|
||||
|
||||
---
|
||||
|
||||
## Action Space
|
||||
|
||||
| Item | Details |
|
||||
| ------------- | ------------------------------- |
|
||||
| **Type** | `Box(-1.0, 1.0, (6,), float32)` |
|
||||
| **Dimension** | 6 |
|
||||
|
||||
The joints correspond as follows:
|
||||
|
||||
| Index | Action Meaning (Joint Position Change) | Min Value | Max Value | Corresponding XML Name |
|
||||
| ----: | --------------------------------------- | :-------: | :-------: | :--------------------: |
|
||||
| 0 | Joint1 (Base Rotation) Position Change | -1 | 1 | `Joint1` |
|
||||
| 1 | Joint2 (Upper Arm) Position Change | -1 | 1 | `Joint2` |
|
||||
| 2 | Joint3 (Forearm) Position Change | -1 | 1 | `Joint3` |
|
||||
| 3 | Joint4 (Wrist Rotation) Position Change | -1 | 1 | `Joint4` |
|
||||
| 4 | Joint5 (Wrist Pitch) Position Change | -1 | 1 | `Joint5` |
|
||||
| 5 | Joint6 (Wrist Rotation) Position Change | -1 | 1 | `Joint6` |
|
||||
|
||||
---
|
||||
|
||||
## Observation Space
|
||||
|
||||
| Item | Details |
|
||||
| ------------- | -------------------------------- |
|
||||
| **Type** | `Box(-inf, inf, (25,), float32)` |
|
||||
| **Dimension** | 25 |
|
||||
|
||||
The observation space consists of the following parts (in order):
|
||||
|
||||
| Part | Content Description | Dimension | Remarks |
|
||||
| ----------- | ----------------------------------------------- | --------- | -------------------------------------------------------------------------------- |
|
||||
| **dof_pos** | Position information for each degree of freedom | 13 | First 6 are arm joints, last 7 are ball's free joint (3 position + 4 quaternion) |
|
||||
| **dof_vel** | Velocity information for each degree of freedom | 12 | Velocity is derivative of position |
|
||||
|
||||
| Index | Observation | Min Value | Max Value | XML Name | Type (Unit) |
|
||||
| ----- | --------------------------- | --------- | --------- | ----------- | ------------------------- |
|
||||
| 0-5 | Arm Joint Angles | -Inf | Inf | Joint1-6 | Angle (rad) |
|
||||
| 6 | Ball x-coordinate | -Inf | Inf | ball_x | Position (m) |
|
||||
| 7 | Ball y-coordinate | -Inf | Inf | ball_y | Position (m) |
|
||||
| 8 | Ball z-coordinate | -Inf | Inf | ball_z | Position (m) |
|
||||
| 9-12 | Ball Orientation Quaternion | -Inf | Inf | ball_qw/xyz | Quaternion (w,x,y,z) |
|
||||
| 13-24 | Joint and Ball Velocities | -Inf | Inf | - | Velocity/Angular Velocity |
|
||||
|
||||
---
|
||||
|
||||
## Reward Function Design
|
||||
|
||||
The reward function consists of the following components:
|
||||
|
||||
```python
|
||||
# Position Control Reward: Keep ball above paddle center
|
||||
# Controlled Upward Velocity Reward: Reward moderate upward velocity when ball is well-positioned
|
||||
# Height Accuracy Reward: Ball is close to target height
|
||||
# Consecutive Bounces Reward: Reward consecutive successful bounces
|
||||
# Total Reward = Weighted combination of all components
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Initial State
|
||||
|
||||
- **Arm Initial Position**: [0, 40, 110, 0, -60, 0] degrees, with random noise
|
||||
- **Ball Initial Position**: Above paddle center, with random noise
|
||||
- **Ball Initial Velocity**: [0.0, 0.0, 0.0] m/s
|
||||
|
||||
---
|
||||
|
||||
## Episode Termination Conditions
|
||||
|
||||
- **Ball Falls**: Ball z-coordinate < 0.05m (near ground)
|
||||
- **Ball Too High**: Ball z-coordinate > target height + 1.0m (lost control)
|
||||
- **Horizontal Deviation Too Far**: Absolute value of ball x-coordinate > 1.5m
|
||||
|
||||
---
|
||||
|
||||
## Usage Guide
|
||||
|
||||
### 1. Environment Preview
|
||||
|
||||
```bash
|
||||
uv run scripts/view.py --env bounce_ball
|
||||
```
|
||||
|
||||
### 2. Start Training
|
||||
|
||||
```bash
|
||||
uv run scripts/train.py --env bounce_ball
|
||||
```
|
||||
|
||||
### 3. View Training Progress
|
||||
|
||||
```bash
|
||||
uv run tensorboard --logdir runs/bounce_ball
|
||||
```
|
||||
|
||||
### 4. Test Training Results
|
||||
|
||||
```bash
|
||||
uv run scripts/play.py --env bounce_ball
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Training Results
|
||||
|
||||
1. Consecutive Bouncing: Capable of achieving 3 or more consecutive bounces
|
||||
2. Position Control: Ball's horizontal position (x-coordinate) stable within target position ± 0.05m range
|
||||
3. Height Control: Ball's height stable within target height 0.8 ± 0.1m range
|
||||
4. Velocity Control: Ball's upward velocity maintained within reasonable range (0.1-1.5 m/s)
|
||||
5. Stable Control: Capable of maintaining stable bouncing for 20 seconds without dropping
|
||||
Reference in New Issue
Block a user