Skip to content

ModelMeta: expose chat template, special tokens, and full GGUF metadata#286

Merged
bernardladenthin merged 1 commit into
bernardladenthin:mainfrom
vaiju1981:model-introspection-extras
Jul 1, 2026
Merged

ModelMeta: expose chat template, special tokens, and full GGUF metadata#286
bernardladenthin merged 1 commit into
bernardladenthin:mainfrom
vaiju1981:model-introspection-extras

Conversation

@vaiju1981

Copy link
Copy Markdown

Summary

Enriches the existing getModelMetaJson native method and the ModelMeta wrapper with three
read-only introspection fields, so callers can inspect a loaded model without a second round-trip:

  • chat_template — the model's resolved Jinja template (ModelMeta.getChatTemplate())
  • special_tokens — bos/eos/eot (plus sep/nl/pad in the JSON) via getBosTokenId() /
    getEosTokenId() / getEotTokenId()
  • metadata — the full GGUF key/value map via getMetadata(key) (e.g. "general.architecture"),
    capped at 2 KB/value so large array metadata (tokenizer tokens/merges) can't bloat the payload

Why

ModelMeta already exposes n_ctx_train / n_embd / n_vocab / n_params / size /
architecture / name. These three fill the common remaining gaps: callers frequently also need
the model's own chat template (to drive prompting), its special-token IDs (chat/FIM), and arbitrary
GGUF metadata — all of which previously required going outside the binding.

Design

  • No new native method or JNI signature change — the fields ride the existing getModelMetaJson
    JSON payload, and the new ModelMeta getters default cleanly when a field is absent (so a mismatched
    older native lib still works).
  • The special-token map is shared between the normal and vocab-only paths via a small
    special_tokens_json helper.
  • The 2 KB/value metadata cap is deliberate — scalar metadata fits; huge array values truncate rather
    than dumping the whole tokenizer into the JSON.

Testing

  • ModelMeta unit tests (JSON-based, no model): 11 total, green — 2 new cover the added getters
    and their absent-field defaults.
  • Native: libjllama builds and links against b9842 and loads cleanly (NativeLibraryLoadSmokeTest).
    Field population over a real GGUF is exercised by CI's model-backed suite.

Enrich the existing getModelMetaJson native method and the ModelMeta wrapper
with three read-only introspection fields, so callers can inspect a loaded
model without a second round-trip:

- chat_template  - the model's resolved Jinja template (getChatTemplate)
- special_tokens - bos/eos/eot (plus sep/nl/pad in the JSON) via
  getBosTokenId/getEosTokenId/getEotTokenId
- metadata       - the full GGUF key/value map via getMetadata(key), capped at
  2 KB per value so large array metadata (tokenizer tokens/merges) cannot bloat
  the JSON

No new native method or JNI signature change: the fields ride the existing
getModelMetaJson payload, and absent fields default cleanly on the Java side.

Verified: libjllama builds and links against b9842, loads cleanly
(NativeLibraryLoadSmokeTest), and the ModelMeta unit tests pass (11 total).
@bernardladenthin

Copy link
Copy Markdown
Owner

Hey again @vaiju1981! Great PRs. I'll move now many files to integrate the longchain as artifact. May wait some hours with new code changes. It will be hard to merge :)

@bernardladenthin bernardladenthin merged commit 93f48c0 into bernardladenthin:main Jul 1, 2026
30 of 36 checks passed
bernardladenthin pushed a commit that referenced this pull request Jul 1, 2026
Pre-existing formatting violation in special_tokens_json (the token-id map added
by #286 / ModelMeta) that this branch inherited unchanged through the move. The
reactor PR's clang-format check surfaced it. Reflowed with the pinned
clang-format 22.1.5; no logic change. (Implies main currently carries the same
violation — it self-resolves when this merges.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Rt1paYztGJ2AKUuBuAGDXE
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants