Skip to content

mlx-reason-ct 0.6.0

Choose a tag to compare

@sandovaljoseph sandovaljoseph released this 10 Nov 23:17
· 9 commits to main since this release

Breaking changes

bfloat16_fast is the default precision, and the float32 and bfloat16_fp32 profiles are removed. NV-Reason-CT is a BF16 model and both removed profiles already read its BF16 weights; asking for them now raises InvalidInputError. bfloat16 remains for NVIDIA's BF16 arithmetic reproduced step by step.

-generate_report("ct.nii.gz", "outputs", model_dir="models", precision="float32")
+generate_report("ct.nii.gz", "outputs", model_dir="models")

Without a prompt, reports are requested in the NV-Reason-CT model card's words, so the report generated without a prompt changes. The request is now "Write a structured chest CT report." or "Write a structured abdominal CT report." instead of "write a structured chest CT report" or "write a structured abdomen CT report", and "Write a structured CT report." without a region. This is the same request mc-nv-reason-ct sends on CUDA.

Fixes

Prompts with tools now match NV-Reason-CT's processor token for token. 0.4.x sorted the schema keys and escaped <, >, & and ', which lengthened the prompt and could change the answer. Prompts without tools are unchanged.

Tool-call response choices containing <tool_call> pass the new response check. The check in the v0.5.0 tag, which was not published to PyPI, rejected them, so reason-ct-tools stopped at its first decision.

If writing an output fails partway, the previous report, response and run files are restored instead of leaving a mixed set.

Report, response and run files are written with 0600 permissions, and a symlink at an output path is replaced instead of followed.

A missing or malformed mlx/manifest.json raises InvalidInputError in mlx-reason-ct verify, instead of FileNotFoundError or JSONDecodeError.

Faster

Prompt processing takes half as long. On an M5 Max with a LUNA16 chest CT, prefill takes 6.2 s instead of 11.9 s and a full report 12.4 s instead of 17.9 s, with the same report text. Measurements

Peak memory drops from 14.4 GB to about 12.5 GB. Visual weights are released once the CT is encoded, and prompt processing runs in bounded blocks. medmlx users get 0.6.0 with medmlx 0.7.0.