Skip to content

v0.9.2 - Consolidation and Refactor, Reasoning support and External Prompts

Choose a tag to compare

View all tags
@danielebruneo danielebruneo released this 17 Aug 06:02
· 1 commit to main since this release
v0.9.2
c59b542

KeepRoLLMing v0.9.2

KeepRoLLMing 0.9.2 is a major internal consolidation release: the proxy now has a canonical streaming path, modular filters, structured observability, stronger end-to-end coverage, and a cleaner public onboarding experience.

Highlights

  • Canonical streaming pipeline

    • One authoritative streaming implementation for SSE parsing, accounting, finalization, serialization, and terminal framing.
    • Preserves OpenAI-compatible ordering for reasoning, tool calls, finish_reason, and [DONE].
    • Improved cancellation and keepalive handling for long-running streams.

  • Modular filter architecture

    • Filters now live under keeprollming.filters, with each module owning its request/stream behavior and configuration validation.
    • Route configuration uses the simpler canonical filters: format.
    • Built-in modules include system prompts, summarization, multimodal validation, tool rewriting, timestamps, model nudge, tool-loop stopping, and reasoning-loop stopping.

  • Observability and debugging

    • Event-based PLAIN and JSON logging with clearer transcript rendering.
    • Per-request performance events and cache-aware throughput metrics.
    • Opt-in raw SSE and request-body capture for focused production debugging.

  • More reliable tool and reasoning flows

    • Correct handling of streamed tool-call fragments and terminal tool_calls finish reasons.
    • Nudge continuations preserve the original assistant prefix and retry with the intended request payload.
    • Tool/reasoning loop safeguards are covered by dedicated regression tests.

  • Configuration and routing

    • New config.example.full.yaml documents the complete supported configuration surface.
    • System prompts can be loaded from files.
    • Route-level reasoning_effort overrides support LibreChat/Qwen-style controls.
    • Public runtime prompt templates are now shipped under prompts/.

  • Onboarding and developer experience

    • Canonical local launcher: ./krm serve, ./krm start, ./krm status.
    • Verified Python/venv setup flow and deterministic fake-backend quick start.
    • Updated README, configuration, deployment, troubleshooting, and contributor documentation.

Validation

  • 1480 tests passed
  • 27 intentional integration skips
  • Clean clone validation completed successfully:
    • fresh virtualenv setup
    • both configuration examples validated
    • fake-backend quick start verified end-to-end

Upgrade notes

This release intentionally removes legacy internal paths and obsolete launcher scripts. Use:

bash scripts/setup.sh
./krm serve --port 8000 --config config.yaml

For configuration, prefer the new canonical filters: route section and use
[config.example.full.yaml](https://github.com/danielebruneo/KeepRoLLMing/blob/v0.9.2/config.example.full.yaml)
as the complete reference.

See the README and configuration guide
for setup and migration details.

Assets 2
Loading