Body
Track one body's pose into control lanes: torso position, hand positions, arm spread, lean, crouch and presence.
- Sensing
- Since 0.2.0
- body
- pose
- sensing
Body tracks one person's pose and turns it into control lanes.
What it does#
Connect a video source to video. The control output has 11 lanes from 0 to 1, in order: x, y, z, hand.l.x, hand.l.y, hand.r.x, hand.r.y, spread, lean, crouch, present. It runs a MediaPipe model in the browser. The model downloads after the person consents, with its size shown first, and is cached afterwards. Nothing is uploaded. When tracking is lost the lanes hold their last values and present falls to 0 after 220 ms.
When to use it#
Use it for whole-body control: arm spread, lean and crouch. See Sensing.
Ports#
| Port | Direction | Signal | Lanes |
|---|---|---|---|
video | Input | Video | 1 |
control | Output | Control | 11 |
Parameters#
| Parameter | Label | Range | Default | Unit | Scale | Notes |
|---|---|---|---|---|---|---|
model | Model | 0 to 2 | 0 | linear | Smaller is faster; larger is steadier. Choices: 0 Lite, 1 Full, 2 Heavy. | |
mirror | Mirror | 0 to 1 | 1 | linear | Flip x for a mirrored picture. Choices: 0 As captured, 1 Mirrored. | |
smooth | Smooth | 0 to 1000 | 80 | ms | linear | One-pole smoothing of every lane. |
delegate | Compute | 0 to 1 | 0 | linear | Run the model on the GPU or the CPU. Choices: 0 GPU, 1 CPU. |
Model (model)#
Chooses the pose model; larger values trade speed for accuracy.
Mirror (mirror)#
Flips x so movement matches a mirror when on.
Smooth (smooth)#
Smoothing time in milliseconds for the lanes.
Compute (delegate)#
Chooses where the model runs: 0 for the default, 1 for the alternative.
Tips#
- Use Select lanes with
"audio": {"channels": 1, "firstChannel": N}to pick one lane; lanes count from 1. - A one-lane control cable can drive any parameter through its
{ "node": "x", "port": "@param" }socket.
See also#
Select lanes, Scale, Smooth, Sensing, Control lanes.
Help patches
Each help patch is an ordinary Toy that teaches one idea. The diagram shows the Patch; the preview shows its Frame. Open it in TuneToys to play and edit it.
1. Arm spread filter
intermediate Teaches: model, mirror, smooth, sensing
Arm spread sweeps a filter.
Connections as text
- Camera video output connected to Body video input
- Body control output connected to Select lanes control input
- Select lanes control output connected to Filter @cutoff input
- Oscillator audio output connected to Filter audio input
- Filter audio output connected to Patch outputs audio input
- Lane 8 of the sensor's control output drives cutoff through its @cutoff socket.
- The model downloads after you agree, then stays cached. On lost tracking the last values hold and present falls.
- ctl.select picks one lane: firstChannel counts from 1.
Related nodes
- Select lanes Extract a contiguous lane range from a control bundle. Use several to split a sensor's lanes.
- Scale Map a control range onto another with a curve, clamp and inversion. Depth and offset for any modulation.
- Smooth Slow a control lane down with separate rise and fall times so jumpy sensors and buttons stop clicking.