Repository navigation
FastGen v0.2.0
FastGen v0.2.0 is the first major update since v0.1.0 and includes every pull request merged since then (#10, #25–#29, #33, #37 and #38). Some import paths, config fields and inference flags changed; see "Upgrading from v0.1.0" below.
Highlights
- Methods: AYF, PDD, AnyFlow, Phased DMD, Causal-Forcing and backward simulation.
- Networks: FLUX.2, Wan 2.2 A14B, Cosmos 3, LingBot-world, and FastGen's first video+audio models: LTX-2, LTX-2.3, LTX-2.5 and MiniMax H3.
- Training core: LoRA for any network, one guidance API for sampling and training, sharded EMA under FSDP2, and a separate precision for method and sampling state.
- Tooling: ComfyUI custom nodes, a VLM judge, FlashDreams streaming inference, and 117 experiment configs, up from 73.
New methods
Trajectory matching now groups the consistency-model and knowledge-distillation methods (CM, ECT, sCM, TCM, MeanFlow, KD, CausalKD) and adds:
- AYF (
AYFModel): continuous-time flow map distillation. - PDD (
PDDModel): parallel decoding distillation for image and video generation.
Distribution matching adds:
- AnyFlow (
AnyFlowModel, #25): any-step video generation with on-policy flow map distillation. - Phased DMD: few-step distribution matching with score matching within timestep subintervals.
- Causal-Forcing: autoregressive diffusion distillation for real-time interactive video.
- Backward simulation: the student trains on its own sampling trajectories instead of noised data.
Fine-tuning adds teacher forcing for causal models (CausalTeacherForcingSFTModel). Causal video methods share autoregressive student sampling with KV caching.
New networks
| Data | New in v0.2.0 |
|---|---|
| Image | FLUX.2 [dev] and FLUX.2 [klein] base 4B and 9B, each for text-to-image and image editing |
| Video | Wan 2.2 A14B (mixture of experts) for text- and image-to-video, Cosmos 3, and LingBot-world for camera-controlled generation |
| Video+audio | LTX-2, LTX-2.3 and LTX-2.5, with image-to-video recipes for LTX-2 and LTX-2.3; MiniMax H3 |
fastgen/networks/README.md lists the supported checkpoints.
Training core
- Module preparation. Modules are prepared once, in the same order on every rank (precision, train mode, gradients, then device placement or FSDP2 sharding), including tied parameters and shared submodules.
- LoRA.
model.net_adapteradds PEFT adapters and extra trainable modules to any network. The teacher reuses the student's frozen base weights instead of keeping a second copy. Adapters train with DDP; FSDP is not supported yet. - Guidance.
model.guidance_cfgcombines classifier-free, mix and skip-layer guidance, each schedulable over time, with optional rescaling. Sampling and teacher passes during training share it. - Sampling grids.
model.student_grid_cfgsets the student's sampling grid (type, steps and options), replacingstudent_sample_stepsandsample_t_cfg.t_list. - Precision.
precisionnow covers only data and method/sampling arithmetic, so it can differ from the model weights (precision_model). Optimizer state, master weights and EMA useprecision_opt, which replacesprecision_fsdp. - FSDP2. EMA copies are sharded like the live network instead of being replicated on every GPU, and validation samples come from the EMA with FSDP too.
model.extra_fsdp_modulesshards frozen modules such as the teacher or text encoder, and meta-device initialization (model.meta_init) also works without FSDP. - torch.compile.
model.torch_compile_modecompiles the training networks (#28). - Environment. A
.envfile (see.env.example) sets the output, data, checkpoint and cache roots. Credentials for W&B, Hugging Face, S3 and the VLM endpoint live in onecredentials_dir.
Tooling
- ComfyUI: custom nodes for teacher and student inference, with an example Qwen-Image and LTX-2.3 workflow (
comfyui/workflows/QwenImage-LTX2.3.json). - VLM judge: the
VLMLoggercallback andscripts/inference/vlm_infer.pyscore generated images and videos with any OpenAI-compatible endpoint. - Inference: video with audio, image editing, camera-controlled generation from pose files, and FlashDreams streaming inference for Self-Forcing Wan 2.1 T2V (
scripts/inference/flashdreams_inference.py). - Data: WebDataset loaders for video+audio and for camera-conditioned video. The
VipeLoggercallback plots camera trajectories and, with the optional ViPE package, reports the pose error of generated clips. - Checkpoints and data: FSDP checkpoint conversion (
scripts/fsdp_convert.py) and resumable S3 downloads (scripts/resumable_s3_download.py). - CI: lint, type checks, tests and package installation run on pull requests (#29).
Bug fixes
Training and checkpoints
- Run names default to the config file name, so recipes in the same group, such as the EDM CIFAR-10 ones, no longer share a
debugdirectory and resume each other's checkpoints. - Resume handles interrupted saves and S3, keeps the FSDP2 optimizer state when some parameters don't match instead of resetting it (#27), and raises an error when a requested checkpoint file is missing.
trainer.max_iter=Nruns N iterations, not N−1.- EMA: bf16 recipes keep their EMA in fp32 so it no longer stalls (a low-precision EMA now warns), and the Wan CausVid recipe updates its EMA.
Methods
- sCM: correct JVP dropout masks, finite-difference JVP, VP timesteps and loss normalization.
- DMD2 and LADD: R1 perturbs the noised input, not the clean data.
- KD: CausalKD no longer crashes, and single-step KD scales its noise correctly.
- TCM and f-distill: no NaN losses in TCM, and f-distill validation keeps its training statistics.
- VSD loss uses higher precision in bf16 recipes.
- Wan MeanFlow: the
rpathway trains (#42).
Networks
- FLUX.1: correct time shift, and no double guidance in FLUX.1-dev.
- SD1.5 and CogVideoX: an empty caption no longer blanks the whole batch.
- SDXL micro-conditioning uses the actual resolution.
- DiT-XL/2 SFT uses DiT's DDPM schedule.
- Wan: local checkpoints load their own components, I2V uses per-checkpoint defaults, and teachers honor the step count.
- VACE: the depth annotator no longer runs with random weights, and causal sampling keeps the caller's noise.
- CogVideoX-5B schedule and RoPE no longer depend on the model path.
- Cosmos Predict2: video2world timesteps (#26), Predict2.5 SFT inference (#33),
reset_parameters, feature extraction and padded-prompt attention masks (#37), and skip-layer guidance and video2world sampling (#38).
Data loading and scripts
- WebDataset: deterministic training resumes at the right sample, and
shard_endandshards_per_workerwork as documented. - Loaders fail fast on unreadable ignore-index files and all-failing streams, and skip malformed scores.
- Inference and FID: latent recipes (
enable_preprocessors=False) work, SFT checkpoints use their CFG, and a wrong--ckpt_pathraises an error.
Smaller fixes
Crashes in DMD2 and Self-Forcing under FSDP2 with precision_fsdp="float32", LambdaLinearScheduler, return_logvar (SD1.5, SDXL, CogVideoX, Cosmos Predict2) and EDM2 with MeanFlow's r; the Adam fallback for an unknown optim_type; EDM2 FID reference statistics; GIF output from --save_high_quality; and download_data.py partial outputs and ImageNet-64 --only-models.
Upgrading from v0.1.0
Imports
fastgen.methods.consistency_modelandfastgen.methods.knowledge_distillationare nowfastgen.methods.trajectory_matching. Imports fromfastgen.methods, such asfrom fastgen.methods import CMModel, still work.fastgen.utilsno longer re-exports helpers:LazyCall,instantiateandlocateare infastgen.configs.instantiate, which replacesfastgen.utils.registry. Custom configs should usefrom fastgen.configs.instantiate import LazyCall as L.expand_likeis infastgen.utils.tensor.set_global_varsandset_temp_global_varsare infastgen.utils.global_vars.
fastgen.networks.inceptionmoved toscripts/fid/inception.py.
Config fields
| v0.1.0 | v0.2.0 |
|---|---|
model.guidance_scale |
model.guidance_cfg.cfg.scale |
model.skip_layers, and skip_layers_start_percent in network sample() |
model.guidance_cfg.cfg.skip_layers, scheduled with cfg.skip_layers_boundary |
model.student_sample_steps |
model.student_grid_cfg.steps |
model.sample_t_cfg.t_list |
model.student_grid_cfg.grid_type="explicit" with grid_kwargs={"timesteps": [...]} |
model.fsdp_meta_init |
model.meta_init |
model.precision_fsdp |
model.precision_opt |
model.add_teacher_to_fsdp_dict=False |
model.extra_fsdp_modules=[] |
trainer.tf32_enabled |
trainer.tf32_matmul and trainer.tf32_conv |
log_config.wandb_credential, trainer.checkpointer.s3_credential |
files in credentials_dir (default ./credentials) |
MeanFlow model.guidance_mixture_ratio=κ |
model.guidance_cfg.mix.scale=κ+1 |
MeanFlow model.guidance_t_start, model.guidance_t_end |
per-interval scale lists split at boundary in model.guidance_cfg |
MeanFlow model.sample_t_cfg.r_sample_ratio=p |
model.sample_t_cfg.flow_matching_ratio=1-p |
MeanFlow model.sample_r_cfg |
model.sample_r_overrides, the sample_t_cfg fields that differ for r |
CM and sCM sample_t_cfg.quantize, sCM loss_config.divide_x_0_spatial_dim, Self-Forcing enable_gradient_in_rollout |
removed |
TF32 matmuls are now off by default, which is PyTorch's own default; v0.1.0 turned them on. Set trainer.tf32_matmul=True to get the old behavior back.
Network arguments
| v0.1.0 | v0.2.0 |
|---|---|
SD1.5, SDXL, FLUX.1 and Qwen-Image model.net.model_id |
model.net.model_id_or_local_path |
EDM model.net.model_type and UNet arguments such as img_resolution, label_dim and dropout |
a UNet config in model.net.backbone, such as L(SongUNet)(img_resolution=32, ...) |
DiT model.net.num_classes |
model.net.label_dim |
Wan model.net.use_wan_official_sinusoidal |
removed |
Networks now reject unknown arguments, so a stale field raises an error instead of being ignored.
Configs and files
EDM/config_dmd2_test.pyis nowEDM/config_dmd2_fast.py.CogVideoX/config_kd.pyandCogVideoX/config_sft_5b.pywere removed.scripts/inference/prompts/validation_aug_qwen_2_5_14b_seed42.txtis nowvideo_prompts.txt.
Inference scripts
--num_stepsis now--sample_steps. It counts denoising steps (sampling-grid intervals), not grid points, and defaults to each network's own step count instead of 50.--fpsis now the config overridemodel.net.fps; usemodel.teacher.fpsfor a separate teacher.--save_as_gifis now--vid_format gif.--num_conditioning_framesand--conditional_frame_timestepwere removed:- Video2World is configured on the network with
model.net.is_video2worldandmodel.net.num_conditioning_frames; seeCosmosPredict2/config_*_v2w.py. - The conditioning-frame timestep is now the
conditional_frame_timestepargument of the Cosmos Predict2 network'sforward()andsample(). It defaults to1e-4; the v0.1.0 flag defaulted to 0.
- Video2World is configured on the network with
Requirements
- Python 3.10–3.12, with NumPy 1.x.
- PyTorch 2.10, diffusers 0.40.0 and transformers 5.15.0 (previously PyTorch ≥2.6, diffusers 0.35.1 and transformers 4.49.0).
- New dependencies include torchvision 0.25, torchao 0.16 and peft 0.20.0.
- The Docker image is based on
nvcr.io/nvidia/pytorch:25.11-py3(previously 25.06). - ARM64: install the CUDA build of PyTorch from
https://download.pytorch.org/whl/cu130beforepip install -e ., as the README describes; PyPI has only CPU builds for aarch64. - Gated models: FLUX.1-dev, FLUX.2-dev, FLUX.2-klein-base-9B and LTX-2.5 are gated on Hugging Face. Accept each model's license there, and provide an access token in
HF_TOKENorcredentials/hf_token.txt.
Known issues
- GPU
torch.stft, which the LTX-2 audio path uses, can fail withCUFFT_INTERNAL_ERRORon some driver and container combinations. We saw this withnvcr.io/nvidia/pytorch:25.11-py3on driver 580.178.04 (GB300); thetorch 2.10.0+cu130wheel from download.pytorch.org works on the same machines. - NumPy 2 is not supported yet (#43).
Released Models
- FastGen-AYF: FLUX.1-dev and Qwen-Image LoRA adapters, distilled with AYF for 4–8-step generation.
Full changelog: v0.1.0...v0.2.0