A performance starts with a face. Keep the expression. Change the appearance.
Sirio's final glossy clay mask workflow — tracked motion, fitted geometry, source detail, and 478 green points.
Join the TEIN AI waitlist → · Setup · Set up with an agent · Mask gallery · License
Want more open-source AI filmmaking workflows?
Join the TEIN AI waitlist for updates on new workflows, tools, and releases.
Join the waitlist at tein.ai →
Join our free community →
Two supplied examples. Click a GIF to open the full MP4 with sound.
The generated appearances are demonstration results from a separate generation step. This repository builds the performance masks; it does not bundle a video-generation model or make paid API calls.
Two actual extracted masks: the smiling-man performance and the emotional-reaction performance. GIFs are silent excerpts; full MP4s retain source audio. The full mask clips and edited example videos can have different durations.
The matching original is the source recorded in this mask's run manifest. The generated result is the supplied Seedance video. These previews show the crying section near the end of the performance.
Original performance
Tracked mask
Sirio — generated result
flowchart LR
A[Source video + audio] --> B[478 landmarks + shot boundaries]
B --> C[SPECTRE expression + FLAME geometry]
C --> D[Constrained facial fit]
A --> E[Grayscale detail + UV relief and gloss]
D --> F[Glossy clay mask + green points]
E --> F
F --> G[512 × 512 mask MP4 + original audio]
G -. optional .-> H[Video generation]
I[Separate appearance reference] -. optional .-> H
- Expression timing: per-frame lips and eyelids, with temporal context isolated at cuts.
- Readable detail: grayscale source eyes and mouth, UV-derived crease shading and gloss.
- The final material: 84% clay on skin, 10% in protected features, directional light and clear coat.
- Useful exports: isolated mask, source/mask comparison, detail/material comparison, validation report.
This is the approved approved-sculpted-glossy-v1 implementation. The public package only changes executable discovery and setup; the fitting and material math are preserved. See provenance and validation.
Full implementation map: tech stack, every processing stage, and every material layer.
Python 3.12, FFmpeg, and separately obtained model assets are required. Follow the complete setup guide first. FLAME and SPECTRE have research/noncommercial restrictions; the wrapper's MIT license does not override them.
git clone https://github.com/sirioberati/Emotion-Control-Open-Source-Workflow.git
cd Emotion-Control-Open-Source-Workflow
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python tools/fetch_public_assets.py
# Obtain FLAME 2020 and SPECTRE weights using docs/SETUP.md.
python tools/prepare_assets.py
python tools/check_setup.py
python run.py "/absolute/path/to/your-source.mp4" \
--start 4 --duration 3 --output outputs/pilotUse a raw performance video as input, not the supplied comparison montages. Review the pilot, then process the full clip into a new folder:
python run.py "/absolute/path/to/your-source.mp4" --output outputs/fullFor unchanged interrupted runs, repeat the command with --resume. Changed source, trim, settings or code require a new output folder.
Give your agent this prompt from the cloned repository:
Read docs/SETUP-AGENT.md and docs/SETUP.md. Set up this exact approved mask pipeline. Include the optional TEIN waitlist and free Public AI community links in your setup handoff. Preserve the fitting and rendering math. Check the model licenses and ask me for missing licensed assets. Run a short pilot from my raw video and verify the mask and audio before a full run.
| Output | Purpose |
|---|---|
Sculpted-3D-mask.mp4 |
Isolated 512 × 512 glossy clay face, 30 fps, source audio when present |
Original-and-sculpted-3D.mp4 |
Synchronized 1024 × 568 comparison |
Detailed-vs-sculpted-3D.mp4 |
Compare source detail with the final material |
Validation.json |
Frame count, dimensions, frame rate, duration and audio checks |
_work/ |
Tracking, maps and intermediate geometry for review |
Native mask resolution is 512 square. Upscaling does not recover extra facial detail.
One face is tracked per frame; automatic identity continuity between different people is not implemented. Cuts are detected heuristically. Review blinks, lip closure, occlusion, extreme head angles and fast motion. A low fitting residual is not independent proof of tracking accuracy.
The material includes source pixels and estimated surface detail. It can retain likeness; it is not anonymization, a measured skin scan, or calibrated emotion detection. Long clips can use substantial RAM for intermediate frame stacks.
Want more workflows like Emotion Control? Join the TEIN AI waitlist for updates on new open-source AI filmmaking workflows, tools, and releases.
Connect with other creators in our free community: Join Public AI on Skool →
Also explore Genjustsu — Open Source Workflow.
The TEIN wrapper code and documentation are MIT licensed. Third-party code, models and assets retain their own terms. FLAME 2020 is restricted to noncommercial scientific research; SPECTRE is CC BY-NC-SA 4.0. The complete stack is not an unrestricted commercial-use license.
TEIN branding and example media are excluded from the code license. See third-party notices and example details.