Digital Human Koubo · AI 数字人口播

One photo + one sentence → a finished talking-head video with a digital human. Script, lip-synced AI presenter, voice clean-up, captions, and pro-YouTuber-style packaging — fully automated by your AI agent.

English · 简体中文

This 27-second livestream ad has no real presenter. Face, voice, studio, captions and the price animation are all AI. The only input was one sentence: "Use this digital human to make a 30-second livestream ad for the WorkBuddy course — 10 lessons, ¥99, was ¥999." Total cost: 482 Renoise credits.

Click a GIF (or "Watch") to play the full video. Every demo below was produced with this repo.

Livestream sales ad · 9:16 · 27 s Style B + depth-text price ¥999→¥99 ▶ Watch	Product recommendation · 9:16 · 89 s Animated captions, emoji, picture-in-picture ▶ Watch	Text behind the presenter · 5 s Vision person matting, 3-layer composite ▶ Watch

Step-by-step tutorial · 16:9 · 4:36 The digital human teaches how she was made (Style D) ▶ Watch	Digital human, upgraded · 16:9 · 5:55 Two presenters, 6 languages, 4 scenes, cat & dog (Style E) ▶ Watch
Launch breakdown · 16:9 · 39 s Presenter on the side, code B-roll on the right (Style C) ▶ Watch	Finance / tech explainer · 16:9 · 2:21 Charts, documents, chapter cards (Style A) ▶ Watch

Packaging component gallery · 48 components · 2:40 · ▶ Watch

All presenters in the demos are AI digital humans and every video is labelled "内容由 AI 数字人演绎" (performed by an AI digital human). The male presenter is the author's own digital twin, made from his own photo.

• Digital human from one photo — a fixed presenter (face, outfit, voice description, prompt template) stored in persona.json, so every video is the same person. Composite her into any scene (livestream studio, shop, classroom, Kyoto street) with GPT Image first, then make her talk.
• Lip-synced talking clips — the script is split into ≤ 15-second segments and generated with Grok Video 1.5 through Renoise MCP. Other models (Wan 3.0, OmniHuman for your own recording) are documented.
• Cost gate — the agent always quotes segments · seconds · credits · balance first and waits for your "yes". It generates segment 1 for you to check before batching the rest.
• Quality control — every segment is transcribed with whisper and compared with the script, and 4 frames per segment are inspected (Grok sometimes burns subtitles into the picture; those segments get re-shot).
• Voice unification — 1/3-octave spectral matching, de-essing, one shared small-room reverb and loudness normalisation, so 30 separately-generated clips sound like one recording (sibilance spread 13.2 dB → 5.9 dB in our tests).
• 5 editing styles, chosen for your content — the agent diagnoses your content, recommends 2 styles with preview images and asks you everything in one go (style · ratio & length · caption intensity · presenter).
• 48 packaging components — measured frame-by-frame from 25 videos by top creators (MrBeast, Mark Rober, Ryan Trahan, MKBHD, MediaStorm/影视飓风, 何同学, Johnny Harris, Vox, Kurzgesagt, 回形针, 半佛仙人, 差评, 毕导). Each style has a default packaging recipe; add or remove techniques by just asking.
• Text behind the presenter — big words / numbers between the background and the person (macOS Vision person segmentation → transparent video → 3-layer composite).
• Multi-language — English, Japanese, Korean, French, Spanish, Arabic tested (whisper back-check 94–100 %).
• Works with your agent — a standard SKILL.md Agent Skill plus scripts; any agent that can load skills, connect a remote MCP server and run local commands can drive it.

your content ──► 0. setup check (Renoise MCP / ffmpeg / whisper — skip what you already have)
                 1. diagnose content → recommend 2 of 5 styles → ask once → brief.json
                 2. script.json (≤15 s per segment, ~5.4 Chinese chars/s)
                 3. quote credits → wait for "yes" → generate segment 1 → you check → batch
                 4. range-download (CDN-safe) → whisper back-check → frame check
                 5. voice unification (spectral match + de-ess + shared room)
                 6. word-level timeline (tl.json)
                 7. edit in Remotion: style template + default packaging (+ your requests)
                 8. render → check mid-transition frames → loudness (-14 / -16 LUFS)

The skill is a folder with a SKILL.md (the open Agent Skills format) plus Python / shell / Remotion scripts. To run the whole pipeline an agent needs three things:

1. Load Agent Skills (a skills folder or skill upload),
2. Connect a remote MCP server (Renoise — streamable HTTP, browser login, no API key),
3. Run commands on your computer (ffmpeg, Python, Node — for downloading, QA and editing).

If your agent can only do MCP + chat (no local commands), you can still use it to plan, quote and generate the clips through Renoise; then run the editing steps with Claude Code / Codex, or by hand with the commands in SKILL.md.

git clone https://github.com/andyhuo520/digital-human-koubo.git
cd digital-human-koubo

The repo contains two skills: digital-human-koubo (main) and finance-explainer (the engine for Style A, the finance / tech explainer).

Claude Code

mkdir -p ~/.claude/skills && cp -R skills/digital-human-koubo skills/finance-explainer ~/.claude/skills/

Codex

mkdir -p ~/.codex/skills && cp -R skills/digital-human-koubo skills/finance-explainer ~/.codex/skills/

WorkBuddy / OpenClaw / other agents — zip the two folders and import them in the product's Skills page, or copy them into its skills directory (see the product's docs):

cd skills && zip -r ../digital-human-skills.zip digital-human-koubo finance-explainer && cd ..

Create an account at renoise.ai first. Login is in the browser — you never paste an API key.

Latest official instructions: renoise.ai/plugin. Check it worked by asking your agent: "查一下我的 Renoise 余额 / check my Renoise balance."

One-command check — it skips what you have and lists install options for what's missing:

bash ~/.claude/skills/digital-human-koubo/scripts/doctor.sh

The text-behind-presenter effect uses Apple Vision and needs macOS; everything else is cross-platform.

The repo ships no third-party audio. Generate an original set of 30 sound effects and a placeholder background track (pure code, no copyright):

python3 ~/.claude/skills/digital-human-koubo/scripts/make_assets.py

new_edit.sh also runs this automatically for every new project. For a better soundtrack, drop any track you have the rights to at <project>/public/music/bgm.mp3.

The default voice is Grok's native voice (best lip-sync). For a designed voice, your own cloned voice, or narration-only videos, the skill uses Xiaomi MiMo TTS:

1. Register at platform.xiaomimimo.com and create an API key. At the time of writing Xiaomi offered the TTS models free for a limited time — check the site for current pricing.
2. Put the key in an environment variable, or in ~/.config/ai-teardown/mimo.env:

   mkdir -p ~/.config/ai-teardown && cat > ~/.config/ai-teardown/mimo.env <<'EOF'
   MIMO_API_KEY=your-key-here
   MIMO_BASE_URL=https://api.xiaomimimo.com/v1
   EOF
   chmod 600 ~/.config/ai-teardown/mimo.env

3. Only clone your own voice or a voice you have explicit permission to use.

Note: Grok generates lip movement from its own voice. Swapping in a different voice afterwards leaves ~100 ms lip offset (fine in picture-in-picture, visible in full-screen close-ups). For your own recording with exact lip-sync, use an audio-driven model such as OmniHuman (documented in the skill).

Talk to your agent in plain language:

用我的数字人 Skill 讲这段：……（paste your content）
Use my digital human skill to make a 30-second livestream ad: WorkBuddy course, 10 lessons, ¥99 (was ¥999).
Make a 3-minute explainer about why Silicon Valley Bank collapsed, Xiaolin-style.
Add a map route when she mentions these countries. / Put "¥99" behind her. / Add a teaser cold open.
把包装组件图鉴发我看看  (show me the packaging gallery)

What happens next:

1. Setup check — anything already installed is skipped.
2. A one-line diagnosis + 2 recommended styles with preview images, and one round of questions.
3. Script + scene design + credit quote → you approve → segment 1 → you approve → the rest.
4. QA, voice clean-up, editing with the style's default packaging → the finished MP4.

1. Pick a photo: frontal face, even light, hair off the face, mouth closed. Only your own photo or one you have permission to use.
2. Ask the agent to generate a 4-view sheet (front, both sides, back) — GPT Image 2.5, 2k, ~23 credits — so the face stays the same from every angle.
3. Compare models with the same image and line (Grok Video 1.5 vs Wan 3.0, 8 s each) and pick one.
4. Upload the reference to Renoise and put its material:<id> into skills/digital-human-koubo/persona.json (renoise.reference_material, or other_hosts.me for a second presenter).

Details: references/new-avatar.md. The material: IDs in this repo belong to the author's account and won't work on yours — that's expected.

Style B has three caption intensities — restrained / standard / lively:

Style A learns only the structure and editing grammar of popular explainer channels. It never uses a real creator's face, voice, set or lines.

We measured 25 videos frame by frame (shot detection, opening beats, on-screen text, sound) — full reports in references/research/.

Rules distilled from the research (packaging-playbook.md):

• Cut count can be low, but stimulation can't stop: in the first 30 s, something new every 3–5 s.
• First 3 s: the claim or the action — no greeting, no logo.
• A BGM bed under everything; sound effects only on events (1–3 per minute in Chinese talking-head channels); 0.6–1.0 s of silence right before a reveal.
• For a single-camera digital human, the cheapest pacing is punch-in cuts on sentence boundaries (1.0× / 1.18× / 1.32×).

Every component is called by name in a pack list, with timings written as "segment:word" (e.g. "3:积分"), and adapts to 9:16 automatically:

"pack": [
  { "use": "NumberSpike", "at": "1:不到十块钱", "until": "1:end", "text": "¥9.9/节", "label": "一节课" },
  { "use": "Danmaku", "items": [{ "t": "真的假的？", "at": "1:" }, { "t": "AI 还能帮我干活？", "at": "1:教你", "hot": true }] },
  { "use": "TextBehind", "src": "main.mp4", "cutout": "main_cut_s4.webm", "cutoutFrom": 17.2, "at": "4:", "until": "4:end",
    "mode": "count", "lines": [{ "t": "¥99", "font": "en", "color": "#FFE14D", "size": 0.17 }], "count": { "from": 999, "to": 99, "prefix": "¥" } }
]

Reference: references/packaging.md. Validate before rendering: python3 scripts/check_edit.py <project>/src.

python3 scripts/cutout_video.py public/main.mp4 public/main_cut_s4.webm --from 17.2 --to 22.4   # matte only the seconds you need

It works on medium shots with headroom. In a selfie close-up the person fills ~89 % of the frame and hides the text — the script warns above 60 %. Mark such segments "shot": "depth" and they are generated as medium shots.

Prices at the time of writing: Grok Video 1.5 — 720p 17 credits/s, 480p 10 credits/s; GPT Image 2.5 2k — 23 credits. Always confirm with the agent's live quote.

skills/
  digital-human-koubo/            main skill
    SKILL.md                      workflow the agent follows
    persona.json                  presenters, voice descriptions, prompt templates
    references/                   style menu · packaging · playbook · install · research reports
    scripts/                      build_tasks · rangedl · assemble · align · check_edit · cutout_video
                                  voicefx/unify · doctor · new_edit · make_assets
    templates/
      huazi-916/                  9:16 data-driven talking-head template (edit.json)
      packaging/                  48-component kit + gallery project
      reference/                  launch-169 · tutorial-169 · multihost-169
    assets/                       style previews · packaging gallery · avatar reference
  finance-explainer/              Style A engine (episode.json beat sheet, research helpers)
docs/                             README images, GIF previews, posters

• Grok occasionally burns subtitles into landscape clips even when told not to → frame check + re-shoot that segment.
• whisper writes Chinese numbers as digits (九百九十九 → 999), so accuracy drops but the line is correct → listen before re-shooting.
• Renoise's US CDN can truncate parallel downloads → rangedl.py downloads in 6 ranges with resume + decode check.
• Grok 9:16 720p is 720×1280 — never stretch it to another ratio.
• Expo easing never quite reaches the target (99.9 → rounds to 100) → counters snap to the exact value at the end.
• Always mute the presenter video track; doubled audio a few ms apart sounds harsh (comb filtering).

• Use only your own face/voice or ones you have explicit permission to use. Never impersonate a real person or creator.
• Every video carries an "AI digital human" label; scene and animal clips are marked "AI generated".
• No made-up numbers, prices, offers or reviews — sample data on screen must be labelled as such.
• Don't use web memes, film/TV stills or celebrity images — the meme components use original stickers generated from your own presenter.
• Credits are spent only after you approve the quote.

• Code and docs: MIT © 2026 Berryxia (andyhuo520).
• Remotion is used for rendering and has its own license: free for individuals and companies of up to 3 people; larger companies need a company license.
• Animated emoji: Google Noto Emoji Animation (CC BY 4.0) — credit "Animated emoji: Google Noto" in your videos.
• Maps: world-atlas (ISC; data from Natural Earth, public domain) · d3-geo, topojson-client (ISC). For maps involving China's borders in formally published content, check against applicable local standards.
• Speech recognition: whisper.cpp (MIT). Generation runs on Renoise (your account, your credits); optional TTS by Xiaomi MiMo.
• Course screenshots in the WorkBuddy demo come from the author's own course. Demo videos are the author's own work; please don't re-upload them as your own.

See THIRD_PARTY.md for details.