docs: refresh CLAUDE.md for grammar/structured-output, perplexity, KVarN defaults - #439
Conversation
…arN defaults - Document the Grammar/ subsystem in Core (ITokenConstraint, JsonSchemaOutputConstraint, per-family tool-arg constraints, #423/#425) - Add the perplexity CLI command and -j/--json-schema structured-output flags - Note --tq-mode auto default (KVarN where supported, Lloyd-Max fallback, #436) and KVarN CPU/CUDA decode paths (#180) - Add SnapKvEval benchmark project; grammar coverage in Tests.Core Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QNixcRevA5NPyFsKUeCySr
There was a problem hiding this comment.
Code Review
This pull request updates the CLAUDE.md documentation to reflect several new features, including perplexity evaluation over a corpus, whole-turn structured JSON-schema output, grammar-constrained decoding, and the new auto mode for TurboQuant KV cache compression. The review feedback suggests correcting the spelling of 'honours' to 'honors' in the CLI documentation to maintain consistency with the American English spelling used throughout the rest of the codebase.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| 3. **Engine** (`SharpInference.Engine`) — Forward-pass orchestration, KV cache, sampling, speculative decoding, MoE expert offloading, continuous batching. Depends on Core + backends. | ||
| 4. **Frontends** — | ||
| - **CLI** (`SharpInference.Cli`, `Spectre.Console.Cli`, llama.cpp-compatible flags): `RunCommand` (default text/vision inference), `ImageCommand` (`image` subcommand), `ListMetadataCommand` (`list-metadata`), `ListTensorsCommand` (`list-tensors`). | ||
| - **CLI** (`SharpInference.Cli`, `Spectre.Console.Cli`, llama.cpp-compatible flags): `RunCommand` (default text/vision inference; also whole-turn JSON-schema structured output via `-j`/`--json-schema`/`--json-schema-file`), `ImageCommand` (`image` subcommand), `PerplexityCommand` (`perplexity` — accuracy gate over a corpus, honours `--tq`/`--tq-mode`), `ListMetadataCommand` (`list-metadata`), `ListTensorsCommand` (`list-tensors`). |
There was a problem hiding this comment.
To maintain spelling consistency across the codebase, please use the American English spelling honors instead of the British English spelling honours. The rest of the codebase consistently uses American English (e.g., honored in Sampler.cs and RunCommand.cs, and honor in RunCommand.cs).
Suggested change:
- **CLI** (`SharpInference.Cli`, `Spectre.Console.Cli`, llama.cpp-compatible flags): `RunCommand` (default text/vision inference; also whole-turn JSON-schema structured output via `-j`/`--json-schema`/`--json-schema-file`), `ImageCommand` (`image` subcommand), `PerplexityCommand` (`perplexity` — accuracy gate over a corpus, honors `--tq`/`--tq-mode`), `ListMetadataCommand` (`list-metadata`), `ListTensorsCommand` (`list-tensors`).
Summary
Updates the checked-in
CLAUDE.mdto reflect features added since it was last written. Documentation-only change — no code touched.Changes
Core/Grammar/directory:ITokenConstraint,JsonSchemaOutputConstraint(whole-turn JSON-schema output), the per-family tool-argument constraints, andToolSchemaCompiler. AddsITokenConstraintto the central interfaces list and notes theSampler's per-step constraint masking.perplexityCLI command — addsPerplexityCommand(KVarN accuracy gate, Implement KVarN variance-normalized KV-cache quantization (2-bit values) #180) to the command examples and the CLI frontend description.-j/--json-schema/--json-schema-file/--json-schema-orderedonRunCommand, with a usage example and the note that the server exposes the same viaresponse_format:json_schema.--tq-mode auto(fix(tq): default --tq-mode to auto — KVarN where supported, Lloyd-Max fallback with quality warning (#432) #436/TurboQuant Lloyd-Max 3-bit collapses on Qwen3 (PPL 61x, degenerate decode) — pre-existing on master #432/Implement KVarN variance-normalized KV-cache quantization (2-bit values) #180) — clarifies theautodefault (KVarN where supported, Lloyd-Max fallback with a quality warning) and the CPU (AVX2) + CUDA-graph decode paths.SnapKvEvalbenchmark project — added to Samples & Scripts; grammar coverage noted in the Tests.Core table row.Test plan
Documentation-only; no build or test impact.
🤖 Generated with Claude Code
Generated by Claude Code