Skip to content

v1.0.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 23 Oct 11:10
· 314 commits to main since this release
v1.0.0
fa9388e
This commit was signed with the committer’s verified signature.
xycld xy_cloud
GPG key ID: 75D0BF0817A8A8A6
Verified
Learn about vigilant mode.

MossInfer 1.0.0: the first stable release of the native C++/CUDA inference engine for MOSS-Image. It needs no PyTorch or Python at run time.

Highlights

  • Memory planning: the engine plans against the GPU memory that is actually free, with three modes: min_memory, balanced and max_speed. When the model doesn't fit, it streams DiT blocks; a 1024² image runs in about 2.3 GB of VRAM.
  • Reference tier: exact bf16, matching PyTorch. Per DiT step it is 1.33–1.39× faster than PyTorch (measured on an RTX 5090). On a 32 GB card a full 1024² image is 1.45× faster than the official pipeline running fully resident; on 16 GB cards, where the official pipeline has to offload, the engine is 2–3.5× faster.
  • Quantized models: INT8 ConvRot and INT4 ConvRot DiT files, INT8 and W4A8 text encoders (ComfyUI / community formats), and 8-bit attention.
  • Runtime LoRAs: kohya, PEFT / diffusers, LyCORIS LoHa / LoKr and full deltas, applied per generation.
  • Generation: 9 samplers and 7 schedulers, img2img, inpainting, a hires fix, distill bundles, latent previews, and PNG metadata.
  • Interfaces: a C API (libmossinfer, ABI 1) and the moss-server JSON-lines sidecar.
  • Packages: Windows x64 and Linux x86-64, each with CUDA 12 and CUDA 13 builds. Each comes as a full package (bundled cuBLASLt) and a small -system-cuda package that uses an installed CUDA.
  • Command line only in 1.0: control LoRAs (colour / depth maps) and the lossy accelerators (CFG truncation, FBCache step cache). They come to the C API and the server in 1.1.

Notes

  • No quality certificates ship in 1.0. auto therefore runs the exact Reference tier for the bf16 model. Choose high or turbo explicitly for the quantized speed tiers; those runs are reported as uncertified. RTX 50 certificates are in progress.
  • GPU support: NVIDIA compute capability 8.x and 12.x (RTX 30, RTX 40, RTX 50, A100, L4/L40). GPU runs are verified on Linux. Windows builds are compiled and CPU-tested in CI.

Verification of this build

  • CI: builds, packages and CPU tests pass on all four flavours.
  • GPU test suite: 410/410 on an RTX 5090 (sm_120). The bit-exact subset, 11/11, passes on an RTX 4090 Laptop (sm_89).
  • With the new features off: output is byte-identical to the previous main in 5 configurations.

See docs/integration.md for integrating the engine and SHA256SUMS to verify the downloads.

Full Changelog: https://github.com/xycld/owoinfer/commits/v1.0.0

Assets 15
Loading