common: add --log-jsonl - #28437
Conversation
github-actions
Bot
added
documentation
|
Nice, one thing before I approve: level_str() has no case for GGML_LOG_LEVEL_NONE, so every LOG() line (the raw llama-cli output and most of the server summaries) comes out as "level":"unknown", it should map to something explicit like "none". Though NONE is not really a log, it's the user-facing output, so does it deserve its own "type" rather than a level under "log"? |
not sure either, but let's still format it as json and give it a type anyway, so that downstream app can simply filter out type != "log" if they don't want to read any log in next PRs, we will progressively add other |
| | `--reasoning-budget N` | token budget for thinking: -1 for unrestricted, 0 for immediate end, N>0 for token budget (default: -1)<br/>(env: LLAMA_ARG_THINK_BUDGET) | | ||
| | `--reasoning-budget-message MESSAGE` | message injected before the end-of-thinking tag when reasoning budget is exhausted (default: none)<br/>(env: LLAMA_ARG_THINK_BUDGET_MESSAGE) | | ||
| | `--reasoning-preserve, --no-reasoning-preserve` | preserve reasoning trace in the full history, not just the last assistant message (default: template default)<br/>compatible with certain templates having 'supports_preserve_reasoning' capability<br/>example: https://docs.z.ai/guides/capabilities/thinking-mode#preserved-thinking<br/>(env: LLAMA_ARG_REASONING_PRESERVE) | | ||
| | `--reasoning-preserve, --no-reasoning-preserve` | preserve reasoning trace in the full history, not just the last assistant message (default: enabled)<br/>compatible with certain templates having 'supports_preserve_reasoning' capability<br/>example: https://docs.z.ai/guides/capabilities/thinking-mode#preserved-thinking<br/>(env: LLAMA_ARG_REASONING_PRESERVE) | |
There was a problem hiding this comment.
Is this being changed on purpose here? It doesn't seem directly related to --log-jsonl.
There was a problem hiding this comment.
llama-gen-doc needs to be run regularly, but people sometimes forget to do it. So, as soon as we run it, we update everything.
There was a problem hiding this comment.
Ah, understood. Thanks for explaining!
3 participants
Overview
Per discussion about equivalent of HF CLI
--jsonoutput, we want to eventually allow downstream apps to read output fromllama-serveras JSON.This is the first step of the plan, allow normal log lines to output as jsonl, with a specific
"type": "log"on each log line.In the future, we can extend this logic to have other types of log output, for example
--fitoutput can have a specific output schema with its owntypeRequirements