Hands
Track up to two hands from a video stream into control lanes: position, depth, pinch, spread, openness and presence per hand.
- Sensing
- Since 0.2.0
- hands
- tracking
- sensing
Hands tracks up to two hands from a video stream and turns them into control lanes.
What it does#
Connect a video source to video. The control output has 14 lanes from 0 to 1, in order: l.x, l.y, l.z, l.pinch, l.spread, l.open, l.present, r.x, r.y, r.z, r.pinch, r.spread, r.open, r.present. It runs a MediaPipe model in the browser. The model downloads after the person consents, with its size shown first, and is cached afterwards. Nothing is uploaded. When tracking is lost the lanes hold their last values and present falls to 0 after 220 ms.
When to use it#
Use it for hands-in-the-air control of a filter, volume or pitch. See Sensing.
Ports#
| Port | Direction | Signal | Lanes |
|---|---|---|---|
video | Input | Video | 1 |
control | Output | Control | 14 |
Parameters#
| Parameter | Label | Range | Default | Unit | Scale | Notes |
|---|---|---|---|---|---|---|
mirror | Mirror | 0 to 1 | 1 | linear | Flip x so the person's right hand is on the right of a mirrored picture. Choices: 0 As captured, 1 Mirrored. | |
smooth | Smooth | 0 to 1000 | 40 | ms | linear | One-pole smoothing of every lane. |
delegate | Compute | 0 to 1 | 0 | linear | Run the model on the GPU or the CPU. Choices: 0 GPU, 1 CPU. |
Mirror (mirror)#
Flips x so the left hand on screen reads as your left when on.
Smooth (smooth)#
Smoothing time in milliseconds for the lanes.
Compute (delegate)#
Chooses where the model runs: 0 for the default, 1 for the alternative.
Tips#
- Use Select lanes with
"audio": {"channels": 1, "firstChannel": N}to pick one lane; lanes count from 1. - A one-lane control cable can drive any parameter through its
{ "node": "x", "port": "@param" }socket.
See also#
Select lanes, Scale, Smooth, Sensing, Control lanes.
Help patches
Each help patch is an ordinary Toy that teaches one idea. The diagram shows the Patch; the preview shows its Frame. Open it in TuneToys to play and edit it.
1. Left hand opens the filter
beginner Teaches: mirror, smooth, sensing
The left hand's x position sweeps a filter.
Connections as text
- Camera video output connected to Hands video input
- Hands control output connected to Select lanes control input
- Select lanes control output connected to Filter @cutoff input
- Oscillator audio output connected to Filter audio input
- Filter audio output connected to Patch outputs audio input
- Lane 1 of the sensor's control output drives cutoff through its @cutoff socket.
- The model downloads after you agree, then stays cached. On lost tracking the last values hold and present falls.
- ctl.select picks one lane: firstChannel counts from 1.
Related nodes
- Select lanes Extract a contiguous lane range from a control bundle. Use several to split a sensor's lanes.
- Scale Map a control range onto another with a curve, clamp and inversion. Depth and offset for any modulation.
- Smooth Slow a control lane down with separate rise and fall times so jumpy sensors and buttons stop clicking.