docs(QUANT-EXL3): align support and usage with code - #3176
Draft
localai-org-maint-bot wants to merge 3 commits into
Draft
docs(QUANT-EXL3): align support and usage with code#3176localai-org-maint-bot wants to merge 3 commits into
localai-org-maint-bot wants to merge 3 commits into
Conversation
The public EXL3 description retains obsolete limits and attempt history. Define a source-checked editorial correction without changing runtime or benchmark claims. Tracks ISSUE-LOCAL-01M2CBGG169HDX0DWKHQWCBBGN. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:GPT-6 [Codex]
The EXL3 feature row mixes current capabilities with obsolete attempts. Describe the CUDA dispatch and artifact limits against the current source. Preserve removed prose in the dated archive and keep artifact pins intact. Issue: ISSUE-LOCAL-01M2CBGG169HDX0DWKHQWCBBGN Branch: row/QUANT-EXL3-DOCS-IMPLEMENT Review: mudler#3176 Focused documentation gates pass, including 19 README checker tests. The operator owns broad preflight and its environment-failure comparison. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6 [Codex]
The MTP width applies to trellis weights, not companion tensors. The ROCm record includes a later BF16 control, so name the remaining clock-attribution and discrete-GPU gaps instead. Issue: ISSUE-LOCAL-01M2CBGG169HDX0DWKHQWCBBGN All seven focused documentation gates pass, including 19 README tests. Artifact filenames, sizes, revisions, and hashes remain unchanged. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Codex:gpt-6 [Codex]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Row
QUANT-EXL3, public documentation only.Tracks
ISSUE-LOCAL-01M2CBGG169HDX0DWKHQWCBBGNin the canonical local issue file.Before starting
e030f1b90. The existing EXL3 implementation claim owns source and its implementation spec. This contribution owns public documentation only.QUANT-EXL3atACTIVE. No implementation lifecycle changes..agents/specs/exl3-public-docs.md.What changed
Replace the old EXL3 news with the current CUDA Qwen3.8-27B and DFlash2 capability. Shorten the EXL3 feature summary and five artifact entries. Correct the obsolete DeepSeek-V4 loader and tokenizer limitations while keeping its real-artifact gate open. Preserve filenames, sizes, revisions, and hashes, and archive removed prose verbatim. Link benchmark conditions without adding speed or correctness claims.
Evidence
Independent final review: PASS, no findings, on
bdfc4d9b038991fea23a634bf4b8172ce1d8ee72. The submitted head has the identical tree. The operator reran the focused gate.scripts/agent-preflight.shpasses. The broad local run exited 1 with failures and skips. Missing CMake/readelf/numpy and extracted Python affect this container; shell/tool suites also fail. The run captured a stale origin/main base before refresh. The record, release-workflow, trailer, and compile-scope checks were rerun successfully against the correct base. No full-suite green is claimed.python3 scripts/check-readme-structure.pypython3 scripts/check-supported-models.py(44 registered architectures)python3 scripts/check-quickstart-recipes.pypython3 scripts/check-benchmark-index.pypython3 scripts/check-agent-record.pypython3 -m unittest discover -s tests/scripts -p 'test_check_readme_structure.py'(19 tests)python3 scripts/check-tree-compiles.py --base upstream/main: no source, header, or build file is in scope.python3 scripts/check-commit-trailers.py --range upstream/main..HEAD --filledandcheck-commit-style.py --range upstream/main..HEAD.e030f1b90: all five edited artifact entries retain their first five cells byte-for-byte.Speed claims
Honest gaps
This is documentation-only work on a CPU-only host. No GPU execution, model inference, or new runtime measurement was performed. Source checks and retained evidence support the descriptions. The sampled HumanEval comparison is explicitly not a correctness gate. Full local preflight is not green, and hosted CI must supply the unavailable tooling. The local issue remains open until the PR lands. No upstream merge is requested.
FOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Codex:GPT-6 [Codex]