[feat]: native Apple MLX studio with Liquid Glass onboarding - #14
[feat]: native Apple MLX studio with Liquid Glass onboarding#14aryan5v wants to merge 15 commits into
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…decode - mx.compile the DiT forward by default (--no-mlx-compile to opt out); bit-identical to eager with an eager fallback, ~1.4x faster denoise. mlx_dit_from_diffusers_safetensors gains a compile= passthrough. - Encode prompts with UMT5 in bf16 by default (--text-encoder-dtype): fp32 exponent range removes the T5-family fp16 overflow risk at fp16 memory cost; bf16 embeds are cast to fp32 for the numpy transport so fp16/fp32 callers (the benchmark) are byte-identical. - Decode wan-vae in bf16 by default (--vae-decode-dtype), matching the reference pipeline's effectively-lossless decode default; TAEHV keeps its validated fp16 path and becomes the default backend. - Defaults are now the validated release shape (480x832x81, INT8) and --model-root resolves through the HF cache instead of a pinned snapshot hash (DEFAULT_MODEL_ROOT kept for the bench import). - CI: run test_mlx_compile_parity.py in both MLX smoke jobs so the default path is gated. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BZCNbygNyuXZba1nQNZnFa
…ac callout - Blog draft: restore the launch story (why Mac, stack, QAD retarget, results, roadmap) using only run-2 numbers and the release-record constraints; EMA named as the released checkpoint pending final visual-review confirmation; remaining TODOs are HF links, refreshed timings, review grid, and acknowledgements. - Release record: log the owner's EMA selection (visual review still gates publication), require stock FP16/PTQ reference columns in the review, and flag that recorded timings predate the new defaults. - README: Apple Silicon quickstart callout next to the install section. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BZCNbygNyuXZba1nQNZnFa
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a native macOS SwiftUI application for the FastWan-QAD 1.3B MLX runtime, featuring local video generation, progressive TAEHV previews, and a Python-to-Swift bridge. The feedback highlights critical improvements to concurrency and resource management: resolving a potential out-of-order execution issue in AppModel by replacing unstructured tasks with sequential dispatch, fixing a retain cycle in ProcessDriver's termination handler, avoiding side-effects in SwiftUI by lazy-initializing AVPlayer on view appearance, and ensuring temporary preview files are cleaned up in mlx_wan_prompt_to_video.py using a try...finally block.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| onLine: { [weak self] line in Task { @MainActor in self?.handleGenerationLine(line, id: id) } }, | ||
| onTermination: { [weak self] status in Task { @MainActor in self?.finishGenerationProcess(id: id, status: status) } } |
There was a problem hiding this comment.
Using Task { @MainActor in ... } inside the onLine callback does not guarantee FIFO (First-In-First-Out) execution order on the MainActor. Since onLine is called sequentially from a background thread, spawning unstructured tasks can cause progress updates or the final completion event to be processed out of order, leading to UI glitches or incorrect state transitions. Use DispatchQueue.main.async instead to guarantee that the events are processed in the exact order they are received.
| onLine: { [weak self] line in Task { @MainActor in self?.handleGenerationLine(line, id: id) } }, | |
| onTermination: { [weak self] status in Task { @MainActor in self?.finishGenerationProcess(id: id, status: status) } } | |
| onLine: { [weak self] line in DispatchQueue.main.async { self?.handleGenerationLine(line, id: id) } }, | |
| onTermination: { [weak self] status in DispatchQueue.main.async { self?.finishGenerationProcess(id: id, status: status) } } |
| try await withCheckedThrowingContinuation { continuation in | ||
| let driver = ProcessDriver() | ||
| let outputLock = NSLock() | ||
| var collected: [String] = [] | ||
| do { | ||
| try driver.start( | ||
| executable: executable, | ||
| arguments: arguments, | ||
| currentDirectory: currentDirectory, | ||
| onLine: { line in | ||
| outputLock.withLock { collected.append(line) } | ||
| onLine(line) | ||
| }, | ||
| onTermination: { status in | ||
| // Keep the driver alive until its fast child exits. | ||
| // Without this capture, a one-line command can finish | ||
| // after the local driver has deallocated and its pipe | ||
| // buffer is lost before the continuation resumes. | ||
| _ = driver.isRunning | ||
| let text = outputLock.withLock { collected.joined(separator: "\n") } | ||
| continuation.resume(returning: (status, text)) | ||
| } | ||
| ) | ||
| } catch { | ||
| continuation.resume(throwing: error) | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Capturing driver strongly inside the onTermination closure creates a retain cycle (driver -> process -> terminationHandler -> onTermination -> driver), causing a memory leak. To prevent this, capture driver weakly in the closure, and use withExtendedLifetime(driver) after the await call in runAndCollect to guarantee the driver remains allocated until the process finishes.
let driver = ProcessDriver()
let result = try await withCheckedThrowingContinuation { continuation in
let outputLock = NSLock()
var collected: [String] = []
do {
try driver.start(
executable: executable,
arguments: arguments,
currentDirectory: currentDirectory,
onLine: { line in
outputLock.withLock { collected.append(line) }
onLine(line)
},
onTermination: { [weak driver] status in
guard let driver else {
continuation.resume(returning: (status, ""))
return
}
_ = driver.isRunning
let text = outputLock.withLock { collected.joined(separator: "\n") }
continuation.resume(returning: (status, text))
}
)
} catch {
continuation.resume(throwing: error)
}
}
withExtendedLifetime(driver) {}
return result| private struct VideoSurface: View { | ||
| @State private var player: AVPlayer | ||
|
|
||
| init(url: URL) { | ||
| _player = State(initialValue: AVPlayer(url: url)) | ||
| } | ||
|
|
||
| var body: some View { | ||
| VideoPlayer(player: player) | ||
| .onAppear { | ||
| player.isMuted = true | ||
| player.play() | ||
| } | ||
| .onDisappear { player.pause() } | ||
| } |
There was a problem hiding this comment.
Initializing AVPlayer directly inside the init of VideoSurface is a side-effect that runs every time the view is initialized, even if SwiftUI decides to discard the view or re-evaluate it without changing its identity. To make the view side-effect free and prevent unnecessary resource allocation, initialize the player as an optional @State and instantiate it inside .onAppear.
private struct VideoSurface: View {
let url: URL
@State private var player: AVPlayer?
var body: some View {
Group {
if let player {
VideoPlayer(player: player)
} else {
Color.clear
}
}
.onAppear {
let p = AVPlayer(url: url)
p.isMuted = true
p.play()
player = p
}
.onDisappear {
player?.pause()
player = nil
}
}
}| decode_latents_to_video( | ||
| model_root=args.model_root, | ||
| latents_np=preview_latents, | ||
| output_path=preview_tmp_path, | ||
| fps=args.fps, | ||
| device_arg=args.torch_device, | ||
| dtype_arg=args.torch_dtype, | ||
| backend="taehv", | ||
| taehv_source_path=args.taehv_source_path, | ||
| taehv_checkpoint_path=args.taehv_checkpoint_path, | ||
| taehv_parallel=args.taehv_parallel, | ||
| ) | ||
| preview_tmp_path.replace(preview_path) | ||
| del preview_latents, pred_video_f32 |
There was a problem hiding this comment.
If decode_latents_to_video raises an exception, the temporary file .preview-step-*.tmp.mp4 will be left behind in the preview directory. Wrap the decoding and replacement in a try...finally block to ensure that any temporary files are cleaned up if an error occurs.
| decode_latents_to_video( | |
| model_root=args.model_root, | |
| latents_np=preview_latents, | |
| output_path=preview_tmp_path, | |
| fps=args.fps, | |
| device_arg=args.torch_device, | |
| dtype_arg=args.torch_dtype, | |
| backend="taehv", | |
| taehv_source_path=args.taehv_source_path, | |
| taehv_checkpoint_path=args.taehv_checkpoint_path, | |
| taehv_parallel=args.taehv_parallel, | |
| ) | |
| preview_tmp_path.replace(preview_path) | |
| del preview_latents, pred_video_f32 | |
| try: | |
| decode_latents_to_video( | |
| model_root=args.model_root, | |
| latents_np=preview_latents, | |
| output_path=preview_tmp_path, | |
| fps=args.fps, | |
| device_arg=args.torch_device, | |
| dtype_arg=args.torch_dtype, | |
| backend="taehv", | |
| taehv_source_path=args.taehv_source_path, | |
| taehv_checkpoint_path=args.taehv_checkpoint_path, | |
| taehv_parallel=args.taehv_parallel, | |
| ) | |
| preview_tmp_path.replace(preview_path) | |
| finally: | |
| if preview_tmp_path.exists(): | |
| preview_tmp_path.unlink() | |
| del preview_latents, pred_video_f32 |
aryan5v
changed the title
|
Follow-up |
1835303 to
eea968f
Compare
|
Final native-app polish is now pushed in
Release gate: before public distribution, publish the shared/EMA/RAW archives at the catalog URLs and replace the blank SHA-256 fields. Local developer builds remain usable for current on-device testing; release builds intentionally fail packaging until the assets are immutable. |
Generate 1/factor of the frames and RIFE-interpolate up to --num-frames on Apple Silicon (rife-mlx), with a light unsharp to counter softening. ~2.7x faster denoise at reconstruction MS-SSIM ~0.97. Adds the rife-mlx optional dep, the fast-mode eval, and docs. Validated end-to-end on the 1.3B INT8 QAD model.
…b-mlx' into aryan/fastvideo-mac-studio
|
Fast generation from draft PR #12 is now integrated into the native app in
Validation:
Release asset gate: publish |
Local setup repair validation (macOS 27)Commit Local artifacts restored and mapped without Hugging Face downloads:
Fresh app diagnosis reports Apple Silicon, MLX, Torch, ffmpeg, EMA, RAW, and RIFE ready. The setup and onboarding UI now describe MPS as an optional auxiliary accelerator rather than a runtime requirement. End-to-end packaged-app results:
Validation:
|
Public download preparationCommit Included
Local release artifacts
Validation
External release gates
|
7ed0356 to
7822aed
Compare
2 participants
Summary
Why native
The release runtime is MLX and Metal native. A native app keeps all prompts and media local while supporting direct Metal execution, native files, Share Sheet, completion notifications, sleep prevention, and durable local history. A browser-only version would require a separate WebGPU model port.
Live preview behavior
DMD already produces a full x0 prediction at every step. With
--preview-dir, the MLX example decodes that prediction through TAEHV after each non-final step. The bridge emits a typedpreviewevent only after an atomic rename, so the app never opens a partially written MP4. The final export replaces the preview automatically.The preview path is opt-in. Existing CLI users pay no additional graph or decode cost.
Local artifact mapping
No Hugging Face download is required for development testing. The app detects and migrates to the validated run-2 artifacts already installed on this Mac:
~/models/qad_int8_v2_ema~/mlx-ckpt-cache-qad-v2-ema/int8~/mlx-ckpt-cache-qad-v2/int8The earlier v1 EMA export is intentionally skipped because visual review proved that it collapses to noise. Checkpoint validation matches the native MLX format,
mlx_dit.jsonplusmlx_dit.safetensors. ffmpeg is discovered through either PATH or the managed runtimes imageio binary.End-to-end validation
Packaged app, macOS 27, Apple M4 Max:
A fox runs through a misty pine forest, leaves kicking up behind it.The first live-preview insertion exposed a macOS 27 beta crash in SwiftUI
VideoPlayerinside_AVKit_SwiftUI. The app now uses AppKitsAVPlayerViewthroughNSViewRepresentable; the same preview-to-final reproduction completes without crashing.Automated validation
apps/fastvideo_mac/scripts/test.shapps/fastvideo_mac/scripts/package_app.shFastVideo.appassembled and ad-hoc codesigneduvx pre-commit run --files <changed files>passed yapf, ruff, codespell, PyMarkdown, mypy, and repository checksStack and remaining release gates
This remains a separate stacked PR targeting
aryan/release/fastwan-qad-int8-1.3b-mlxatbec73975. It does not modify the launch branch directly.Remaining public-distribution gates are the published Hugging Face model URL and checksums, Developer ID signing, Hardened Runtime review, and notarization.