[feat]: add workload-driven generation launcher by aryan5v · Pull Request #18 · aryan5v/FastVideo

aryan5v · 2026-07-31T02:39:10Z

Summary

Model-agnostic FastVideo generation and profiling launcher driven by MotionKernel workload manifests.

runs native, optimized, fused, or candidate modes in separate processes
preserves distinct candidate and optimized result artifacts
writes structured timing, actual CUDA peak-memory, environment, frame, and failure metadata
optionally performs a dedicated post-warmup torch.profiler generation
exports metadata-only operator rows without prompts, tensor values, arbitrary environment variables, or executable paths
keeps profiled execution outside clean A/B timing samples

Validation

pytest tests/local_tests/optimizations/test_generation_launcher_workload.py -q — 7 passed
FastVideo pre-commit invocation completed; these example/test paths are intentionally excluded by repository hook configuration
dry-run against Wan/LTX workloads
GPU: real Wan profiler export and MotionKernel ingestion
GPU: LTX launcher/profiler smoke

Example

python examples/inference/optimizations/generation_launcher.py \
  --workload /path/to/motionkernel/workloads/ltx_480p.yaml \
  --mode native \
  --output-dir /tmp/ltx_profile \
  --profile-output /tmp/ltx_profile/profiler.json

Summary by CodeRabbit

New Features
- Added a workload-driven generation launcher supporting native, optimized, fused, candidate, and dry-run modes.
- Added structured result reporting with timings, memory usage, environment details, logs, failures, and optional profiler artifacts.
- Added metadata-only profiling for pipeline calls, including timing, input shapes, and runtime information.
- Added configurable optimization-profile output, skipped runs, workload, model, and task settings.
Documentation
- Documented launcher commands, profiling workflows, and result metadata formats.
Tests
- Added coverage for workload validation, launch modes, profiling, environment handling, and result exports.

Greptile Summary

This PR introduces a workload-driven generation launcher (generation_launcher.py) that runs a single FastVideo generation mode (native, optimized, fused, or candidate) from a versioned MotionKernel YAML/JSON manifest, and a companion worker-side profiler (fastvideo/optimization/profiler.py) that captures a metadata-only operator export from a designated post-warmup pipeline call.

Launcher (generation_launcher.py): loads and validates workload manifests (schema_version 1), resolves mode-specific env vars without leaking them in dry-run, times warmup+measurement runs, saves frames/logs/failure metadata, and writes structured result JSON files with deterministic write-then-replace atomicity.
Profiler (fastvideo/optimization/profiler.py): activated via the FASTVIDEO_OPTIMIZATION_PROFILE_* env vars, wraps the exact target pipeline forward pass with torch.profiler, filters out duplicate CUDA-activity rows, and emits a portable JSON without prompts, tensor values, or executable paths.
Pipeline integration (composed_pipeline_base.py): adds a per-instance _optimization_profile_calls counter so the worker can identify and profile exactly the configured forward pass number.

Confidence Score: 5/5

Safe to merge — all functionality is additive, gated behind new env vars that default to no-ops, and the pipeline change is a transparent wrapper that passes through when profiling is not configured.

The launcher, profiler, and pipeline integration are all new code that leaves existing inference paths unaffected. The previously flagged issues (env var leaks in dry-run, empty-dict fallthrough in resolve_mode_env, repeated torch imports, module-level sys.path mutation) are all resolved in this revision. The key correctness invariants — one profile per designated call index, atomic file writes, no os.environ mutation in dry-run — are directly covered by the test suite.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename	Overview
examples/inference/optimizations/generation_launcher.py	New launcher script; resolve_mode_env uses key-existence checks (not truthiness), correctly handles explicit empty dicts and the fused/optimized/candidate alias chain; dry-run never mutates os.environ; atomic write-then-replace used throughout.
fastvideo/optimization/profiler.py	New worker-side profiler; filters CUDA-activity rows to avoid double-counting, derives per-rank output paths, uses a no-op fast path when profiling is not requested, and is exercised by unit tests with a fake profiler.
fastvideo/pipelines/composed_pipeline_base.py	Adds per-instance _optimization_profile_calls counter and wraps the stage loop in the optimization_profile context manager; no-op when env var is unset so normal inference has negligible overhead.
fastvideo/envs.py	Adds five new FASTVIDEO_OPTIMIZATION_PROFILE_* env var declarations and lambdas; defaults are conservative and consistent with the profiler's logic.
tests/local_tests/optimizations/test_generation_launcher_workload.py	CPU-only tests load the launcher via a module-scoped importlib fixture (no sys.path mutation), use monkeypatch for env isolation, and cover dry-run, mode-env resolution, and CLI delegation.
fastvideo/tests/optimization/test_profiler.py	Uses a fake profiler and monkeypatching to verify skip-call logic, JSON schema, and that no file is written before the target call index.

Sequence Diagram

sequenceDiagram
    participant CLI as generation_launcher CLI
    participant LCH as run_generation()
    participant VG as VideoGenerator (GPU worker)
    participant PPL as ComposedPipelineBase.forward()
    participant PRF as optimization_profile()

    CLI->>LCH: --workload, --mode, --output-dir, --profile-output
    LCH->>LCH: load_workload_dict() — validate schema_version 1
    LCH->>LCH: apply_mode_env() — write mode-specific vars to os.environ
    LCH->>LCH: "set FASTVIDEO_OPTIMIZATION_PROFILE_* env vars (if --profile-output)"
    LCH->>VG: VideoGenerator.from_pretrained(model_id)

    loop warmup runs (0 .. warmups-1)
        LCH->>VG: generator.generate(request)
        VG->>PPL: forward(batch)
        PPL->>PRF: "optimization_profile(call_index < target) no-op yield"
        PRF-->>PPL: pass-through
        PPL-->>VG: ForwardBatch
        VG-->>LCH: result
    end

    opt --profile-output requested
        LCH->>VG: generator.generate(request) [dedicated profiling call]
        VG->>PPL: forward(batch)
        PPL->>PRF: "optimization_profile(call_index == target)"
        PRF->>PRF: "torch.profiler.profile — record shapes & CUDA time"
        PRF->>PRF: _write_export() to profile.json (atomic .tmp replace)
        PRF-->>PPL: exit context
        PPL-->>VG: ForwardBatch
        VG-->>LCH: result
        LCH->>LCH: assert profile_output.is_file()
    end

    loop timed runs (0 .. runs-1)
        LCH->>LCH: cuda.reset_peak_memory_stats + synchronize
        LCH->>VG: generator.generate(request)
        VG->>PPL: forward(batch)
        PPL->>PRF: "optimization_profile(call_index > target) no-op yield"
        PRF-->>PPL: pass-through
        PPL-->>VG: ForwardBatch
        VG-->>LCH: result + timings
    end

    LCH->>VG: generator.shutdown()
    LCH->>LCH: "write_json({mode}_result.json, payload) atomic"
    LCH-->>CLI: exit 0 (ok) or 1 (failed)

_{Reviews (7): Last reviewed commit: "[fix]: honor empty mode overrides" | Re-trigger Greptile}

Add a model-agnostic FastVideo launcher that executes one generation mode from a versioned MotionKernel workload manifest. Writes structured result JSON for native-versus-optimized end-to-end measurement without model-specific callables.

coderabbitai · 2026-07-31T02:39:38Z

📝 Walkthrough

Walkthrough

The pull request adds a workload-driven FastVideo generation launcher and metadata-only optimization profiler. It validates workloads, runs configured generation modes, records results, profiles pipeline calls, and adds tests and documentation.

Changes

Optimization execution

Layer / File(s)	Summary
Workload and request contract `examples/inference/optimizations/generation_launcher.py`, `tests/local_tests/optimizations/test_generation_launcher_workload.py`	The launcher validates versioned workloads, resolves prompts, builds requests, translates runtime settings, and selects mode-specific environment values.
Pipeline profiler integration `fastvideo/envs.py`, `fastvideo/optimization/*`, `fastvideo/pipelines/composed_pipeline_base.py`, `fastvideo/tests/optimization/test_profiler.py`	The optimization profiler reads configuration, profiles selected pipeline calls, collects timing and shape metadata, and writes JSON exports. `ComposedPipelineBase` wraps stage execution with the profiler.
Generation execution and result persistence `examples/inference/optimizations/generation_launcher.py`, `tests/local_tests/optimizations/test_generation_launcher_workload.py`	The launcher applies runtime settings, performs warmups and timed generations, captures measurements and failures, supports profiler output, and writes structured results.
CLI validation and usage `examples/inference/optimizations/generation_launcher.py`, `tests/local_tests/optimizations/test_generation_launcher_workload.py`, `examples/inference/optimizations/README.md`	The CLI supports execution modes, model overrides, dry runs, and workload errors. Tests cover dry-run behavior and mode normalization. Documentation describes commands and result metadata.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant run_generation
  participant VideoGenerator
  participant ComposedPipelineBase
  participant optimization_profile
  participant JSONExport
  CLI->>run_generation: workload and execution mode
  run_generation->>VideoGenerator: create configured generator
  VideoGenerator->>ComposedPipelineBase: execute generation
  ComposedPipelineBase->>optimization_profile: profile pipeline call
  optimization_profile->>JSONExport: write profiling metadata
  run_generation->>JSONExport: write generation result
  JSONExport-->>CLI: result paths and status

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding a workload-driven generation launcher.

✨ Finishing Touches 💡 2

📝 Generate docstrings 💡

Create stacked PR
Commit on current branch

🛠️ Fix failing CI checks 💡

Fix failing CI checks

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch universal-ws1-generation-launcher

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

Prefer exact mode_env keys, keep dry-run from mutating os.environ, import torch once for timing, and load the launcher in tests without rewriting sys.path at import time.

aryan5v · 2026-07-31T02:46:25Z

Addressed Greptile findings:

resolve_mode_env prefers the exact mode key before optimized/fused aliases
--dry-run no longer mutates os.environ (uses resolve-only)
Single import torch for CUDA timing in the measurement loop
Tests load launcher via fixture without permanent sys.path mutation

Left unmerged until review is fully clean.

aryan5v · 2026-07-31T03:59:13Z

GPU validation update:

Wan 2.1 T2V 1.3B completed end-to-end on a GB200 through this launcher (480x832, 49 frames, 4 steps). Clean measured wall times: 4.4263s and 4.3876s; peak allocated CUDA memory: 8707.67 MiB.
The worker produced a metadata-only profiler export and MotionKernel ingested all 2,211 original rows successfully.
Commit 4c7a175 now excludes raw CUDA activity rows so operator totals do not double-count the same kernels alongside CPU-side operator attribution.
Commit d7d5551 adds the two package markers requested by Greptile; directory-level collection now passes (9 tests).
LTX GPU launcher/profiler validation is currently running under SLURM.

greptile-apps · 2026-07-31T04:00:46Z

Want your agent to iterate on Greptile's feedback? Start a greploop in Codex and it will work through the open comments and keep going until this PR reviews clean.

aryan5v · 2026-07-31T04:11:35Z

LTX GPU validation complete (SLURM job 907, exit 0):

Model: FastVideo/LTX2-Distilled-Diffusers
Workload: 480x768, 97 frames, 8 steps, seed 1024
Clean wall samples: 4.7832s, 4.5994s (median 4.6913s)
Clean generation samples: 4.6343s, 4.4972s
Peak allocated CUDA memory: 67,802.22 MiB
Profiler export: 3,071 CPU-side rows, 2,770,568.50 us exclusive CUDA time, zero raw CUDA activity rows
MotionKernel ingestion and ranking completed successfully

This confirms the same launcher/profiler path works unchanged for both Wan and LTX.

[feat]: add workload-driven generation launcher

2d3f207

Add a model-agnostic FastVideo launcher that executes one generation mode from a versioned MotionKernel workload manifest. Writes structured result JSON for native-versus-optimized end-to-end measurement without model-specific callables.

greptile-apps Bot reviewed Jul 31, 2026

View reviewed changes

[fix]: address Greptile findings on generation launcher

2556d71

Prefer exact mode_env keys, keep dry-run from mutating os.environ, import torch once for timing, and load the launcher in tests without rewriting sys.path at import time.

[feat]: export generation profiler metadata

fe29d4c

aryan5v mentioned this pull request Jul 31, 2026

Add universal workload, profiling, and impact-ranking foundation aryan5v/motionkernel#9

Merged

4 tasks

aryan5v added 4 commits July 30, 2026 20:31

[fix]: execute non-dry generation launcher path

3493ee1

[feat]: collect optimization profiles in GPU workers

15e16be

[fix]: avoid profiler CUDA double counting

4c7a175

[test]: package optimization test directories

d7d5551

greptile-apps Bot reviewed Jul 31, 2026

View reviewed changes

Comment thread examples/inference/optimizations/generation_launcher.py

[fix]: honor empty mode overrides

7c2827b

aryan5v mentioned this pull request Jul 31, 2026

[feat]: capture FX graph metadata during optimization profiles #19

Open

coderabbitai Bot mentioned this pull request Aug 2, 2026

Release generic MotionKernel artifact dispatch V1 #26

Open

Navigation Menu

[feat]: add workload-driven generation launcher - #18

[feat]: add workload-driven generation launcher#18
aryan5v wants to merge 8 commits into
mainfrom
universal-ws1-generation-launcher

aryan5v commented Jul 31, 2026 •

edited by greptile-apps Bot

Loading

Uh oh!

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aryan5v commented Jul 31, 2026

Uh oh!

aryan5v commented Jul 31, 2026

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 31, 2026

Uh oh!

aryan5v commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

aryan5v commented Jul 31, 2026 • edited by greptile-apps Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Example

Summary by CodeRabbit

Greptile Summary

Confidence Score: 5/5

Important Files Changed

Sequence Diagram

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

❌ Failed checks (1 warning)

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aryan5v commented Jul 31, 2026

Uh oh!

aryan5v commented Jul 31, 2026

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 31, 2026

Uh oh!

aryan5v commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

aryan5v commented Jul 31, 2026 •

edited by greptile-apps Bot

Loading

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading