Face
Track one face into control lanes: position, head pose, gaze, mouth, smile, blinks, brows and presence.
- Sensing
- Since 0.2.0
- face
- tracking
- sensing
Face tracks one face and turns it into control lanes.
What it does#
Connect a video source to video. The control output has 15 lanes from 0 to 1, in order: x, y, z, yaw, pitch, roll, gaze.x, gaze.y, mouth, smile, blink.l, blink.r, brow, jaw, present. Mouth is lane 9. It runs a MediaPipe model in the browser. The model downloads after the person consents, with its size shown first, and is cached afterwards. Nothing is uploaded. When tracking is lost the lanes hold their last values and present falls to 0 after 220 ms.
When to use it#
Use it to play with your face: open your mouth to open a filter. See Sensing.
Ports#
| Port | Direction | Signal | Lanes |
|---|---|---|---|
video | Input | Video | 1 |
control | Output | Control | 15 |
Parameters#
| Parameter | Label | Range | Default | Unit | Scale | Notes |
|---|---|---|---|---|---|---|
mirror | Mirror | 0 to 1 | 1 | linear | Flip x and yaw for a mirrored picture. Choices: 0 As captured, 1 Mirrored. | |
smooth | Smooth | 0 to 1000 | 60 | ms | linear | One-pole smoothing of every lane. |
delegate | Compute | 0 to 1 | 0 | linear | Run the model on the GPU or the CPU. Choices: 0 GPU, 1 CPU. |
Mirror (mirror)#
Flips x so movement matches a mirror when on.
Smooth (smooth)#
Smoothing time in milliseconds for the lanes.
Compute (delegate)#
Chooses where the model runs: 0 for the default, 1 for the alternative.
Tips#
- The 15-lane output cannot plug straight into a parameter; pick a lane first.
- Use Select lanes with
"audio": {"channels": 1, "firstChannel": N}to pick one lane; lanes count from 1. - A one-lane control cable can drive any parameter through its
{ "node": "x", "port": "@param" }socket.
See also#
Select lanes, Scale, Smooth, Sensing, Control lanes.
Help patches
Each help patch is an ordinary Toy that teaches one idea. The diagram shows the Patch; the preview shows its Frame. Open it in TuneToys to play and edit it.
1. Mouth filter
beginner Teaches: mirror, smooth, sensing
Opening your mouth opens a filter.
Connections as text
- Camera video output connected to Face video input
- Face control output connected to Select lanes control input
- Select lanes control output connected to Filter @cutoff input
- Oscillator audio output connected to Filter audio input
- Filter audio output connected to Patch outputs audio input
- Lane 9 of the sensor's control output drives cutoff through its @cutoff socket.
- The model downloads after you agree, then stays cached. On lost tracking the last values hold and present falls.
- ctl.select picks one lane: firstChannel counts from 1.
Related nodes
- Select lanes Extract a contiguous lane range from a control bundle. Use several to split a sensor's lanes.
- Scale Map a control range onto another with a curve, clamp and inversion. Depth and offset for any modulation.
- Smooth Slow a control lane down with separate rise and fall times so jumpy sensors and buttons stop clicking.