[feat]: preserve Apple MLX future 5B, I2V, and self-forcing tracks

.github/workflows/ci-macos-mlx.yml

@@ -0,0 +1,124 @@
+name: macOS MLX Smoke
+
+on:
+  pull_request:
+    branches: [main]
+    paths:
+      - ".github/workflows/ci-macos-mlx.yml"
+      - "fastvideo/mlx_runtime/**"
+      - "fastvideo/tests/mlx/**"
+      - "fastvideo/platforms/mps.py"
+      - "fastvideo/platforms/__init__.py"
+      - "fastvideo/__init__.py"
+      - "examples/inference/basic/mlx_*.py"
+      - "fastvideo/benchmarks/mlx_*.py"
+      - "pyproject.toml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: macos-mlx-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  mlx-smoke:
+    if: github.event_name == 'workflow_dispatch' || github.event.pull_request.draft != true
+    runs-on: macos-15
+    timeout-minutes: 25
+    env:
+      FASTVIDEO_ATTENTION_BACKEND: TORCH_SDPA
+      TOKENIZERS_PARALLELISM: "false"
+      MASTER_ADDR: localhost
+      MASTER_PORT: "29513"
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+
+      - uses: astral-sh/setup-uv@v3
+
+      - name: Install lightweight MLX smoke dependencies
+        run: |
+          uv pip install --system \
+            --index-url https://download.pytorch.org/whl/cpu \
+            torch==2.11.0 torchvision torchaudio
+          uv pip install --system \
+            pytest numpy scipy pillow imageio einops cloudpickle filelock \
+            PyYAML diffusers huggingface_hub remote-pdb safetensors loguru mlx
+
+      - name: Show Apple runtime
+        run: |
+          python - <<'PY'
+          import platform
+          import mlx.core as mx
+          import torch
+
+          print("machine:", platform.machine())
+          print("processor:", platform.processor())
+          print("mlx default device:", mx.default_device())
+          memory_size = mx.metal.device_info().get("memory_size") if mx.metal.is_available() else "metal unavailable"
+          print("mlx memory_size:", memory_size)
+          print("torch:", torch.__version__)
+          print("torch mps available:", torch.backends.mps.is_available())
+          PY
+
+      - name: Run MLX smoke tests
+        run: |
+          python -m pytest \
+            fastvideo/tests/mlx/test_dmd_sampling.py \
+            fastvideo/tests/mlx/test_memory_limits.py \
+            fastvideo/tests/mlx/test_quant_capability.py \
+            fastvideo/tests/mlx/test_mlx_dit_parity.py \
+            fastvideo/tests/mlx/test_mlx_checkpoint.py \
+            fastvideo/tests/mlx/test_mlx_fastwan_benchmark.py \
+            fastvideo/tests/mlx/test_taehv_decode.py \
+            -q
+
+  # Same tests on MLX's CPU backend. Hosted macOS runners are scarce and
+  # slower to schedule; this Linux job gives fast PR signal on the identical
+  # graph (the parity tests were designed to be backend-agnostic), while the
+  # macOS job above stays the source of truth for Metal behavior.
+  mlx-smoke-linux-cpu:
+    if: github.event_name == 'workflow_dispatch' || github.event.pull_request.draft != true
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    env:
+      FASTVIDEO_ATTENTION_BACKEND: TORCH_SDPA
+      TOKENIZERS_PARALLELISM: "false"
+      MASTER_ADDR: localhost
+      MASTER_PORT: "29513"
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+
+      - uses: astral-sh/setup-uv@v3
+
+      - name: Install lightweight MLX smoke dependencies (CPU backend)
+        run: |
+          uv pip install --system \
+            --index-url https://download.pytorch.org/whl/cpu \
+            torch==2.11.0 torchvision torchaudio
+          uv pip install --system \
+            pytest numpy scipy pillow imageio einops cloudpickle filelock \
+            PyYAML diffusers huggingface_hub remote-pdb safetensors loguru "mlx[cpu]"
+
+      - name: Run MLX smoke tests (CPU backend)
+        run: |
+          python -m pytest \
+            fastvideo/tests/mlx/test_dmd_sampling.py \
+            fastvideo/tests/mlx/test_memory_limits.py \
+            fastvideo/tests/mlx/test_quant_capability.py \
+            fastvideo/tests/mlx/test_mlx_dit_parity.py \
+            fastvideo/tests/mlx/test_mlx_checkpoint.py \
+            fastvideo/tests/mlx/test_mlx_fastwan_benchmark.py \
+            fastvideo/tests/mlx/test_taehv_decode.py \
+            -q

docs/design/apple_silicon_qad_blog_draft.md

@@ -0,0 +1,50 @@
+# FastWan-QAD-INT8-1.3B: Local Text-to-Video on Apple Silicon
+
+> Draft — do not publish until the release owner selects raw or EMA from the
+> final visual review and the model-card checksums are available.
+
+## TL;DR
+
+FastVideo's Apple-native FastWan runtime generates a five-second 480p-class
+text-to-video clip locally on an Apple M4 Max. The release candidate uses an
+MLX INT8 DiT, a three-step DMD schedule, and TAEHV decode. The recorded
+480x832x81 result is 123.7 seconds end to end, including 117.6 seconds of
+denoising, at 5.63 GiB MLX peak memory.
+
+This is a source-install, text-to-video release for the validated M4 Max,
+36 GB-class configuration. It is not a general 16 GB Mac claim, and it does
+not include image-to-video.
+
+## What ships
+
+- FastWan-QAD-INT8-1.3B in Diffusers form plus a pre-quantized MLX DiT
+  checkpoint that avoids requantizing weights at startup.
+- One source-tree generation command using MLX for denoising and MPS for text
+  encoding and TAEHV decode.
+- A reproducible benchmark harness and Metal/MLX-CPU runtime tests.
+
+TAEHV is vendored under its MIT license; FastVideo source is Apache-2.0.
+
+## Results and release decision
+
+The candidate raw and EMA checkpoints measure respectively 0.9360 and 0.9331
+mean MS-SSIM when each INT8 result is compared with the same model's FP16
+result. That metric measures quantization consistency only. It does not score
+absolute visual quality, so it cannot select a release checkpoint.
+
+**Release owner decision: [TODO: raw or EMA after final motion7 visual review].**
+
+The final post must include the selected revision, checksums, model-card link,
+fixed generation command, and the reviewed visual examples. It must not claim
+bitwise fake-quant parity, use the invalid 0.9860 run-1 EMA number, or claim
+that 16 GB Macs have been validated.
+
+## Run it
+
+```console
+uv pip install -e '.[mlx]'
+python examples/inference/basic/mlx_wan_prompt_to_video.py [release command from the model card]
+```
+
+See the Apple Silicon FastWan guide for the full command, hardware statement,
+and troubleshooting.

docs/design/apple_silicon_qad_runbook.md

@@ -0,0 +1,162 @@
+# Runbook: Wan2.1-1.3B → 3-step INT8 QAD on the DGX B200
+
+Operator instructions for launching the Mac-targeted quantization-aware DMD
+distillation (roadmap M4 Phase B) on a DGX with B200 GPUs. Everything below
+runs from a clone of `aryan5v/FastVideo` on branch
+the approved future QAD integration branch. The training recipe is
+`examples/train/configs/distribution_matching/wan/dmd2_t2v_mlx_int8.yaml`:
+frozen Wan2.1-T2V-1.3B teacher + critic, trainable student whose linear
+weights are fake-quantized every forward onto MLX's affine INT8 deploy
+grid (the `mlx_qat` callback; serialized quantizer state is pinned exactly and
+the Metal/PyTorch forward representation is bounded to one source-dtype epsilon by
+`fastvideo/tests/mlx/test_mlx_affine_qat_parity.py`), distilled to the
+3-step FastWan schedule `[1000, 757, 522]`.
+
+Use **4 GPUs** (the recipe default) — do not grab all 8 unless told to.
+Expected wall time for the full run is roughly 4–8 hours on 4×B200, but run
+the smoke test first and extrapolate from its measured seconds/step.
+
+**If only N (< 4) GPUs are free:** the run works on any count ≥ 1; the only
+rule is that the HSDP shard dim must equal the GPU count. Add these overrides
+to every launch command below (shown for N=3):
+
+```bash
+NUM_GPUS=3 bash examples/train/run.sh <config> \
+  --training.distributed.num_gpus 3 \
+  --training.distributed.hsdp_shard_dim 3 \
+  ...
+```
+
+Global batch scales with GPU count (1 per GPU), which is fine for DMD at this
+scale; wall time scales inversely (~6–11 h on 3 GPUs). Prefer waiting for a
+4th GPU only if it frees up within the hour; otherwise just run on 3. The
+same rule applies upward: if 8 are genuinely free and idle, `NUM_GPUS=8` with
+`hsdp_shard_dim 8` roughly halves the wall time.
+
+## 0. Preflight
+
+```bash
+nvidia-smi          # expect B200s; confirm >= 4 idle
+python3 --version   # 3.10–3.12
+```
+
+Verify credentials are present (both are expected to be preconfigured on the
+box — do not write them into the repo):
+
+```bash
+test -n "$WANDB_API_KEY" && echo "wandb ok"
+hf auth whoami || huggingface-cli whoami   # HF auth for model + dataset pulls
+```
+
+## 1. Clone and install
+
+```bash
+git clone https://github.com/aryan5v/FastVideo.git && cd FastVideo
+git checkout aryan/future/fastwan-qad-5b-i2v
+uv venv --python 3.12 && source .venv/bin/activate
+uv pip install -e ".[dev,mlx]"
+```
+
+B200 is sm_100: torch must be a recent CUDA build (the pinned deps are).
+If any attention backend fails to import or pick a kernel at startup, force
+the portable one — correctness is identical, it is only somewhat slower:
+
+```bash
+export FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA
+```
+
+Sanity-check the QAT machinery on this box before spending GPU time:
+
+```bash
+pytest fastvideo/tests/mlx/test_mlx_affine_qat_parity.py -q
+pytest fastvideo/tests/training/test_mlx_qat_callback.py -q
+```
+
+## 2. Dataset
+
+```bash
+python scripts/huggingface/download_hf.py \
+  --repo_id "FastVideo/Wan-Syn_77x448x832_600k" \
+  --local_dir "data/Wan-Syn_77x448x832_600k" \
+  --repo_type "dataset"
+```
+
+This is large (order of a terabyte) — start it in `tmux` early and check free
+disk first (`df -h .`). The recipe reads it from
+`data/Wan-Syn_77x448x832_600k` relative to the repo root.
+
+## 3. Smoke run (mandatory, ~30–60 min)
+
+```bash
+tmux new -s qad-smoke
+NUM_GPUS=4 bash examples/train/run.sh \
+  examples/train/configs/distribution_matching/wan/dmd2_t2v_mlx_int8.yaml \
+  --training.loop.max_train_steps 100 \
+  --training.checkpoint.output_dir outputs/smoke_mlx_int8 \
+  --training.tracker.run_name wan2.1_qad_int8_smoke \
+  --callbacks.validation.every_steps 50
+```
+
+What to verify before proceeding:
+
+1. The log contains a line like `mlx_qat: fake-quantizing N weights (int8,
+   group_size=64, ...)` with N in the hundreds — that is the QAT callback
+   arming. If instead it raises `mlx_qat matched no weights`, stop and
+   report.
+2. No crash referencing `parametrizations` together with FSDP/HSDP/DTensor.
+   The QAT callback wraps weights with torch parametrizations after model
+   setup; a sharding interaction here is the one known integration risk. If
+   it crashes, capture the full traceback and stop — do not work around it
+   by removing the callback (that silently turns the run into plain DMD).
+3. Loss values are finite, W&B run `wan2.1_qad_int8_smoke` is logging under
+   project `distillation_wan`, and the step-50/100 validation clips are not
+   black/NaN garbage (blurry is fine at step 100).
+4. Note the steady seconds/step from the log and extrapolate: full run =
+   4000 × s/step. Report that number back.
+
+## 4. Full run
+
+```bash
+tmux new -s qad-full
+NUM_GPUS=4 bash examples/train/run.sh \
+  examples/train/configs/distribution_matching/wan/dmd2_t2v_mlx_int8.yaml \
+  --callbacks.validation.every_steps 200
+```
+
+Output/checkpoints land in `outputs/wan2.1_dmd2_3steps_mlx_int8`
+(checkpoint every 20 steps, last 3 kept), W&B run
+`wan2.1_dmd2_3steps_mlx_int8`. The job is resumable:
+`--training.checkpoint.resume_from_checkpoint <output_dir>/checkpoint-<step>`.
+Monitor W&B; the DMD generator loss is noisy by nature — judge by the
+validation clips trending sharper/more coherent, not by the loss curve alone.
+
+## 5. Export (1 GPU) — both raw and EMA
+
+Export twice: the raw student, and the EMA shadow weights (usually visibly
+smoother). Both are evaluated on the Mac side; the better one ships.
+
+```bash
+python -m fastvideo.train.entrypoint.dcp_to_diffusers \
+  --checkpoint outputs/wan2.1_dmd2_3steps_mlx_int8 \
+  --output-dir outputs/wan2.1_qad_int8_diffusers \
+  --role student
+
+python -m fastvideo.train.entrypoint.dcp_to_diffusers \
+  --checkpoint outputs/wan2.1_dmd2_3steps_mlx_int8 \
+  --output-dir outputs/wan2.1_qad_int8_ema_diffusers \
+  --role student --ema
+```
+
+Each auto-picks the latest checkpoint and writes a Diffusers-style model dir.
+If the `--ema` export errors, report the traceback and still deliver the raw
+export — it unblocks Mac evaluation while the EMA path gets fixed.
+
+## 6. Deliverables
+
+Report back: (a) the W&B run URL, (b) measured seconds/step and total wall
+time, (c) the path to `outputs/wan2.1_qad_int8_diffusers`, and (d) 2–3
+validation clips from late in training. Mac-side evaluation then happens per
+`docs/design/apple_silicon_fastvideo.md` (M4 exit criteria): load the export
+through the MLX runtime, quantize INT8 on load (the grid the student trained
+on), `--save-mlx-checkpoint`, and run the benchmark suite against the
+INT8-PTQ baseline.

docs/design/fastwan_qad_int8_1_3b_release.md

@@ -0,0 +1,39 @@
+# FastWan-QAD-INT8-1.3B Apple Release Record
+
+This is the committed launch-evidence and publication checklist. Do not commit
+generated videos or visual grids.
+
+## Reproducible evidence
+
+| Field | Recorded result |
+| --- | --- |
+| Hardware | Apple M4 Max, 36 GB-class unified memory (MLX: 38.65 GB) |
+| Runtime | macOS 14+, Python 3.12, MLX 0.31.2 |
+| Shape | 480x832, 81 frames, 3-step DMD (`1000,757,522`) |
+| Path | MPS prompt encode, MLX INT8 DiT, TAEHV decode |
+| Total / denoise / peak | 123.7 s / 117.6 s / 5.63 GiB |
+
+QAD v2 raw and EMA measure 0.9360 and 0.9331 mean MS-SSIM against each
+model's own FP16 output. This is quantization consistency, not absolute quality
+and not a checkpoint-selection metric.
+
+## Mandatory pre-publication gate
+
+1. Fresh source install with `uv pip install -e '.[mlx]'`; retain passing
+   Metal and MLX CPU suite output.
+2. Run `motion7` at the recorded shape; validate all MP4s, timings, memory,
+   Diffusers-to-MLX conversion, and cold pre-quantized-checkpoint reload.
+3. A named release owner must visually select raw or EMA. Neither is selected
+   by this branch; do not publish model or blog first.
+4. Do not claim 16 GB support without a separate pass on a physical 16 GB Mac.
+
+## Model-card and blog requirements
+
+The model card must identify the chosen raw/EMA checkpoint, base provenance,
+exact revision, model artifacts, SHA-256 checksums, fixed generation command,
+hardware/software evidence, intended T2V-only use, and limitations. It must
+include FastVideo's Apache-2.0 notice and TAEHV's MIT notice.
+
+The launch blog may quote the M4 Max measurement. It must not claim physical
+16 GB support, use the invalid 0.9860 run-1 EMA score, call fake-quant parity
+bitwise, claim everything is Apache-2.0, or preselect raw versus EMA.