Motrixlab/docs/source/en/user_guide/demo/shadow_hand_repose.md

Shadow Hand Cube Repose

Overview

This document describes in detail the Shadow Hand dexterous manipulation cube reorientation task environment. This is a classic benchmark test in the field of robotic manipulation, requiring the robot to reorient a cube in-hand to match a randomly sampled target orientation.

:poster: _static/images/poster/shadow_hand_repose.jpg
:nocontrols:
:autoplay:
:playsinline:
:muted:
:loop:
:width: 100%

---

Environment Description

The Shadow Hand cube reorientation task is built based on the realistic Shadow Hand 24-DOF dexterous hand, designed to train robots to rotate a cube in-hand to a target pose through fine multi-finger coordination.

Robot Structure

Shadow Hand is a high-fidelity 24-DOF dexterous hand composed of the following main parts:

• Palm: The base structure of the hand, fixed to the base
• 5 Fingers:
  • Thumb: 5 degrees of freedom, including CMC rotation, MCP, IP joints
  • Index: 4 degrees of freedom, including MCP, PIP, DIP joints
  • Middle: 4 degrees of freedom, including MCP, PIP, DIP joints
  • Ring: 4 degrees of freedom, including MCP, PIP, DIP joints
  • Little: 4 degrees of freedom, including MCP, PIP, DIP joints
• Actuator Configuration: 20 actuated joints (4 coupled joints driven by other joints)

Finger Joint Details

Each finger (except thumb) contains:

• MCP Joint: Metacarpophalangeal joint, 2 degrees of freedom (abduction/adduction + flexion)
• PIP Joint: Proximal interphalangeal joint, 1 degree of freedom (flexion)
• DIP Joint: Distal interphalangeal joint, 1 degree of freedom (flexion)

Thumb contains:

• CMC Joint: Carpometacarpal joint, 2 degrees of freedom
• MCP Joint: Metacarpophalangeal joint, 1 degree of freedom
• IP Joint: Interphalangeal joint, 1 degree of freedom

Environment Objects

• Cube: 50mm * 50mm * 50mm cube
  • Mass: approximately 0.028 kg
  • Friction coefficient: 1.2
  • Initial position: above palm center (0.33, 0.00, 0.295) meters
• Target Visualization: Semi-transparent target pose indicator (mocap body)

Task Goals

The robot needs to complete the following operation goals:

1. Maintain Grasp: Maintain stable grip of the cube in-hand
2. Perceive Goal: Observe target pose (visualization indicator)
3. Fine Manipulation: Rotate cube through multi-finger coordination
4. Pose Alignment: Rotate cube pose to target pose (tolerance <20>0.1 radian)

---

Action Space

Action space is Box(-1, 1, (20,), float32), representing position control commands applied to 20 actuated joints (normalized).

Control Mode

The environment uses position control mode, actions are converted to joint target positions through:

1. Scale from [-1, 1] to actuator control range
2. Optional: action smoothing (moving average filter)
3. Clip to joint limits
4. Apply to simulator actuators

Action Processing Flow

# 1. Scale to actuator limits
targets = scale(actions, lower_limits, upper_limits)

# 2. Action smoothing (optional)
if act_moving_average < 1.0:
    targets = α * targets + (1-α) * prev_actions

# 3. Clip to limits
targets = clip(targets, lower_limits, upper_limits)

# 4. Apply control
actuator_ctrls = targets

Action Dimension Details

Index	Finger	Joint	DOF	Description
0-4	Thumb	J0-J4	5	CMC, MCP, IP
5-8	Index	J0-J3	4	MCP, PIP, DIP
9-12	Middle	J0-J3	4	MCP, PIP, DIP
13-16	Ring	J0-J3	4	MCP, PIP, DIP
17-20	Little	J0-J3	4	MCP, PIP, DIP

---

Observation Space

Observation space is Box(-inf, inf, (157,), float32), containing robot proprioceptive information, cube state, target state, and fingertip state.

Observation Components

Observation vector consists of the following parts (in order):

1. Hand Joint State (48 dimensions)

• Joint Position (24 dims): Unscaled raw joint angles
• Joint Velocity (24 dims): Joint angular velocities scaled by 0.2

2. Cube State (17 dimensions)

• Position (3 dims): Cube position in world coordinates (x, y, z)
• Orientation (4 dims): Quaternion (x, y, z, w)
• Linear Velocity (3 dims): Cube linear velocity
• Angular Velocity (3 dims): Angular velocity scaled by 0.2
• Normalization Factor: Velocity observations multiplied by vel_obs_scale = 0.2

3. Goal State (11 dimensions)

• Goal Position (3 dims): Fixed at (0.33, 0.00, 0.295)
• Goal Orientation (4 dims): Randomly sampled target quaternion
• Relative Rotation (4 dims): Relative quaternion from cube to goal

4. Fingertip State (65 dimensions)

State of 5 fingertips, 13 dimensions per fingertip:

• Position (3 dims): Fingertip position in Cartesian space
• Orientation (4 dims): Fingertip quaternion
• Velocity (6 dims): Linear and angular velocities

Fingertip Link Names:

• rh_ffdistal: Index fingertip
• rh_mfdistal: Middle fingertip
• rh_rfdistal: Ring fingertip
• rh_lfdistal: Little fingertip
• rh_thdistal: Thumb fingertip

5. Action History (20 dimensions)

• Previous action values, for temporal context in the policy

Observation Details

Index	Observation Content	Dimension	Range	Unit
0-23	Hand joint position (unscaled)	24	Joint limits	rad
24-47	Hand joint velocity (0.2)	24	±π/2	rad/s
48-50	Cube position	3	Real	m
51-54	Cube orientation (quat w,x,y,z)	4	Unit norm	-
55-57	Cube linear velocity	3	Real	m/s
58-60	Cube angular velocity (0.2)	3	Real	rad/s
61-63	Goal position	3	Fixed	m
64-67	Goal orientation (quat w,x,y,z)	4	Unit norm	-
68-71	Relative rotation (quat w,x,y,z)	4	Unit norm	-
72-136	Fingertip state (5*13)	65	-	-
137-156	Previous action	20	[-1, 1]	-

---

Reward Function

The reward function uses a composite design with multiple reward and penalty terms.

Main Reward Terms

1. Rotation Alignment Reward (core objective)

   rot_reward = rot_reward_scale / (|rot_dist| + rot_eps)
   

   • Scaling factor: 1.0
   • Epsilon: 0.1
   • Rotation distance calculation: Using quaternion rotation distance formula
   • Incentive: Reward grows inversely as cube orientation approaches target

2. Position Distance Penalty

   dist_reward = dist_reward_scale <20> goal_dist
   

   • Scaling factor: -10.0
   • Distance calculation: Euclidean distance from cube to goal position
   • Incentive: Prevent cube from dropping, keep near target position

3. Action Regularization Penalty

   action_penalty = action_penalty_scale <20> ||actions||<7C>
   

   • Scaling factor: -0.0002
   • Purpose: Encourage smooth, energy-efficient motion

Conditional Rewards

4. Success Reward

   if |rot_dist| d success_tolerance:
       reward += reach_goal_bonus
   

   • Reward value: 2.0
   • Tolerance: 0.1 radian (approximately 5.7<EFBFBD>)
   • Purpose: Sparse reward for achieving goal alignment

5. Drop Penalty

   if goal_dist e fall_dist:
       reward += fall_penalty
       terminated = True
   

   • Penalty value: 0.0 (termination only, no additional penalty)
   • Distance threshold: 0.24 meters
   • Purpose: Terminate episode when cube is dropped

---

Initial State

Hand Initialization

Position Initialization:

The palm is fixed in the world coordinate system, position determined by the model file.

Joint Angle Initialization:

• Use model default joint positions
• Add uniform random noise: [-0.2, 0.2] radians
• Range: All 24 hand degrees of freedom

Velocity Initialization:

All joint velocities are initialized to zero.

Cube Initialization

Position Initialization:

• Fixed position: (0.33, 0.00, 0.295) above palm center
• Add uniform random noise: [-0.01, 0.01] (<28>1cm)

Orientation Initialization:

• Use Shoemake method to generate uniformly distributed random quaternions
• Ensure uniform sampling on SO(3) space

Velocity Initialization:

All linear and angular velocities are initialized to zero.

Goal Initialization

Position Initialization:

• Fixed position: (0.33, 0.00, 0.295) (same as cube initial position)

Orientation Initialization:

• Use Shoemake method to generate uniformly distributed random target quaternions
• Resampled each reset

---

Termination Conditions

Episodes terminate under the following conditions:

1. Drop Termination: Cube distance from goal position e fall_dist (0.24m)
2. Timeout Termination: Reaching max_episode_steps (default 1000 steps)
3. NaN Protection: Detecting rotation distance or position distance as NaN

Success Holding Mechanism

The environment uses a consecutive success counter:

• When rotation tolerance is satisfied, counter increments
• When max_consecutive_successes (50) is reached, trigger success termination and reset goal
• Rotation tolerance: 0.1 radian

---

Usage

Training

uv run scripts/train.py --env shadow-hand-repose

Policy Evaluation

uv run scripts/play.py --env shadow-hand-repose

Environment Visualization

uv run scripts/view.py --env shadow-hand-repose

TensorBoard

uv run tensorboard --logdir runs/shadow-hand-repose

---

Configuration Parameters

Environment Parameters

Parameter	Default	Description
max_episode_seconds	10.0	Maximum episode length (s)
ctrl_dt	0.01	Control timestep (s)
max_episode_steps	1000	Maximum episode steps
num_hand_dofs	24	Total hand DOFs
num_actuators	20	Number of actuated joints

Reward Parameters

Parameter	Default	Description
dist_reward_scale	-10.0	Position distance reward
rot_reward_scale	1.0	Rotation alignment reward
rot_eps	0.1	Rotation reward epsilon
action_penalty_scale	-0.0002	Action regularization
success_tolerance	0.1	Success tolerance (rad)
reach_goal_bonus	2.0	Success reward
fall_dist	0.24	Drop distance threshold
fall_penalty	0.0	Drop penalty

Reset Noise Parameters

Parameter	Default	Description
reset_position_noise	0.01	Cube position noise (m)
reset_dof_pos_noise	0.2	Joint position noise (rad)
reset_dof_vel_noise	0.0	Joint velocity noise (rad/s)

Observation Scaling Parameters

Parameter	Default	Description
vel_obs_scale	0.2	Velocity observation scale

---

References

This environment is based on the following classic works:

• OpenAI Dactyl (2018): First successful in-hand manipulation sim-to-real transfer
• Isaac Gym (2021): High-performance GPU-accelerated physics simulation
• Isaac Lab (2023): Modular robot learning framework