A PoseNet to TouchDesigner integration using Runway ML
Runway is an intuitive machine learning tool for creators.
What's this project
This project is a basic implementation of PoseNet in TouchDesigner via an OSC bridge to Runway. It includes:
1. PoseNet OSC tox
A tox external for communicating with Runway running PoseNet
2. Example project
An example project showing how to use the pose data to outline a face, including calculations of face's distance from the camera.
How to use the .tox
To use the .tox first load it into your project, make sure the OSC port and IP are correct and toggle
this will send the initiation message to Runway and begin streaming detected pose data from the
The output includes the overall pose score as we as a
_y for every keypoint.
Each pose detected, in case the model runs in multipose mode, will be represented as a new row in the
You can also
- choose which keypoints to export (less keypoints improve performance).
- Normalize pose coordinates to screen space (0 to 1) instead of absolute pixel position.
How to setup a Runway environment with PoseNet over OSC.
- Restore a workspace or start from a new model and choose
- Set the output to
OSCand verify the ip and port.
Enable OSCin TouchDesigner.