Sensing
Sensor nodes read a video stream or the device and give back fixed, labelled control lanes from 0 to 1.
Sensing nodes turn what a camera sees, or how a device moves, into control lanes you can plug into any parameter.
What it does#
Hands, Face, Body and Gestures take a video input from a source and emit a fixed, labelled control output with values from 0 to 1. Motion reads the device's accelerometer and gyroscope and has no input. Each lane has a fixed label and position, listed on the node's page.
The tracking nodes run MediaPipe models in the browser. A model downloads only after the person consents, with its size shown, and is cached afterwards. Video and sensor data stay on the device; the Toy never records or uploads them. When tracking is lost the lanes hold their last values and the present lane falls after 220 ms.
A sensor's output is wide, so it cannot go straight into a parameter. Use Select lanes with "audio": {"channels": 1, "firstChannel": N} to pick one lane (lanes count from 1; for example the Face mouth lane is lane 9), then plug that one-lane cable into a parameter socket such as { "node": "filter", "port": "@cutoff" }. Shape it first with Scale or Smooth.
Video needs the browser's WebGPU engine for pictures; sensing runs where the browser supports it. The Player and exports do not run video or sensing yet.
Tips#
- Connecting all lanes straight to a one-lane input is a width conflict and is rejected.
- Use
presentto mute or fade a sound when nobody is in view.
See also#
Video, Control lanes, Select lanes, Smooth.