Motrixlab/docs/source/en/user_guide/demo/dm_finger.md

Finger Manipulation

Finger is a classic manipulation task from the DeepMind Control Suite. A two-link “finger” applies torques to interact with a rotating spinner. MotrixLab currently provides three Finger environments:

:poster: _static/images/poster/dm_finger_spin.jpg
:nocontrols:
:autoplay:
:playsinline:
:muted:
:loop:
:width: 100%

:poster: _static/images/poster/dm_finger_turn.jpg
:nocontrols:
:autoplay:
:playsinline:
:muted:
:loop:
:width: 100%

• dm-finger-spin: make the spinner rotate continuously in the target direction
• dm-finger-turn-easy: align the spinner tip (tip) with a target point (larger target radius)
• dm-finger-turn-hard: same as Turn, but with a smaller target radius

---

Task Description

Finger is a planar (x-z) interaction task:

• The finger has 2 actuated hinge joints: proximal and distal
• The spinner rotates around joint hinge, and tip denotes the spinner tip position
• For Turn tasks, a target point is sampled around the spinner at the beginning of each episode

---

Action Space

Item	Details
Type	Box(-1.0, 1.0, (2,), float32)
Dimension	2

The actions correspond to:

Index	Action Description	Min Control	Max Control	XML Name	Joint Type
0	Torque applied to proximal joint	-1	1	proximal	hinge
1	Torque applied to distal joint	-1	1	distal	hinge

---

Observation Space

MotrixLab follows dm_control-style observations, but flattens them into a single vector.

Spin Observation Space

Item	Details
Type	Box(-inf, inf, (9,), float32)
Dimension	9

The observation vector contains (in order):

• position (4): qpos(proximal, distal) + tip_xz (tip position relative to the spinner in x-z)
• velocity (3): qvel(proximal, distal, hinge) (hinge velocity is used by Spin reward)
• touch (2): log(1 + touchtop), log(1 + touchbottom)

Turn Observation Space

Item	Details
Type	Box(-inf, inf, (12,), float32)
Dimension	12

Compared to Spin, Turn adds:

• target_position (2): target position relative to the spinner in x-z
• dist_to_target (1): signed distance from tip to the target sphere surface (negative means “inside”)

---

Reward Function Design

Spin

In dm_control, Spin is typically defined with a sparse threshold on spinner angular velocity. MotrixLab defaults to a dense/shaped reward for easier training, while also logging the sparse version:

spin_sparse = 1 if hinge_velocity <= -15 else 0
spin = clip(-hinge_velocity / 15, 0, 1)

Turn (Easy / Hard)

Turn aims to bring the spinner tip into a target sphere around the spinner:

• turn_sparse = 1 when dist_to_target <= 0
• MotrixLab defaults to a shaped reward based on distance-to-target (exponential decay), and adds auxiliary terms to reduce “no-contact” failure modes and action jitter:
  • approach-to-spinner shaping
  • touch bonus
  • action magnitude / action change penalties

The final shaped reward is clipped to [0, 1].

---

Initial State

• proximal, distal joint angles are sampled uniformly within joint limits
• spinner hinge angle is sampled uniformly in [-pi, pi]
• for Turn tasks, the target is sampled around the spinner on the x-z plane at reset

---

Episode Termination Conditions

Termination

If NaN appears in the observations

---

Usage Guide

1. Environment Preview (random actions)

uv run scripts/view.py --env dm-finger-spin

uv run scripts/view.py --env dm-finger-turn-easy

uv run scripts/view.py --env dm-finger-turn-hard

2. Start Training

uv run scripts/train.py --env dm-finger-spin --train-backend torch

uv run scripts/train.py --env dm-finger-turn-easy --train-backend torch

uv run scripts/train.py --env dm-finger-turn-hard --train-backend torch

3. View Training Progress

uv run tensorboard --logdir runs/dm-finger-spin

4. Test Training Results

scripts/play.py will auto-discover the latest best_agent.* under runs/{env-name}/ (or you can pass --policy explicitly):

uv run scripts/play.py --env dm-finger-turn-hard

---

Expected Training Results

1. dm-finger-spin: stable continuous rotation in the target direction
2. dm-finger-turn-easy: consistent contact and alignment with the (larger) target region, with reduced jitter
3. dm-finger-turn-hard: successful alignment with a smaller target region, typically requiring more training and better contact behavior