Gestures
Recognise seven hand gestures into one gate lane each, with confidence and presence.
- Sensing
- Since 0.2.0
- gestures
- tracking
- sensing
Gestures recognises seven hand gestures and gives each one a gate lane.
What it does#
Connect a video source to video. The control output has 9 lanes from 0 to 1, in order: closed_fist, open_palm, pointing_up, thumb_down, thumb_up, victory, love, confidence, present. Each gesture lane is a gate. It runs a MediaPipe model in the browser. The model downloads after the person consents, with its size shown first, and is cached afterwards. Nothing is uploaded. When tracking is lost the lanes hold their last values and present falls to 0 after 220 ms.
When to use it#
Use it for switches and triggers made with a hand sign. See Sensing.
Ports#
| Port | Direction | Signal | Lanes |
|---|---|---|---|
video | Input | Video | 1 |
control | Output | Control | 9 |
Parameters#
| Parameter | Label | Range | Default | Unit | Scale | Notes |
|---|---|---|---|---|---|---|
sure | Confidence | 0.3 to 1 | 0.75 | linear | Minimum confidence before a gesture lane goes high. | |
hold_ms | Hold | 0 to 2000 | 120 | ms | linear | How long a gesture must persist before it registers. |
smooth | Smooth | 0 to 1000 | 0 | ms | linear | One-pole smoothing of the confidence lane. |
Confidence (sure)#
Minimum confidence, from 0.3 to 1, before a gesture opens its gate.
Hold (hold_ms)#
Keeps a gate open this many milliseconds after the gesture ends.
Smooth (smooth)#
Smoothing time in milliseconds for the lanes.
Tips#
- Use Select lanes with
"audio": {"channels": 1, "firstChannel": N}to pick one lane; lanes count from 1. - A one-lane control cable can drive any parameter through its
{ "node": "x", "port": "@param" }socket.
See also#
Select lanes, Scale, Smooth, Sensing, Control lanes.
Help patches
Each help patch is an ordinary Toy that teaches one idea. The diagram shows the Patch; the preview shows its Frame. Open it in TuneToys to play and edit it.
1. Open palm gate
intermediate Teaches: sure, hold_ms, smooth, sensing
An open palm fades a tone in.
Connections as text
- Camera video output connected to Gestures video input
- Gestures control output connected to Select lanes control input
- Select lanes control output connected to Oscillator @amplitude input
- Oscillator audio output connected to Filter audio input
- Filter audio output connected to Patch outputs audio input
- Lane 2 of the sensor's control output drives amplitude through its @amplitude socket.
- The model downloads after you agree, then stays cached. On lost tracking the last values hold and present falls.
- ctl.select picks one lane: firstChannel counts from 1.
Related nodes
- Select lanes Extract a contiguous lane range from a control bundle. Use several to split a sensor's lanes.
- Scale Map a control range onto another with a curve, clamp and inversion. Depth and offset for any modulation.
- Smooth Slow a control lane down with separate rise and fall times so jumpy sensors and buttons stop clicking.