Skip to content

v0.1.0 — Portable adaptive gateway

Pre-release
Pre-release

Choose a tag to compare

@midhatn midhatn released this 23 Oct 23:18
· 151 commits to main since this release

Adaptive Inference is now the canonical home for the shared adaptive engine work. This initial release provides an installable gateway in front of existing model servers, while retaining earlier Strata and Hermes integrations as pinned, reconstructable source overlays.

Included

  • A bounded shared queue for user and background agent requests, learned execution-time feedback, and optional observed RAM/Windows-commit admission.
  • Transparent Chat Completions, supported text/embedding routes, model listing and SSE; separate gateway/backend authentication, bounded cancellation and uncertain-completion handling.
  • A caller-validated correction helper that retains the original task and bounds attempts.
  • Eight engine profiles: Strata, llama.cpp, vLLM, SGLang, Ollama, LM Studio, MLX-LM and TGI.
  • Ten agent/client recipes: Hermes, OpenClaw, Aider, OpenHands, Continue, Cline, Roo Code, LangGraph, AutoGen and CrewAI.
  • Pinned dependencies, a no-model HTTP demo, wheel/source packages, cross-platform CI, source provenance, licenses, 33 primary-source bibliography entries and a staged development/upstream plan.

Verified

All 45 gateway/profile/CLI tests and four reconstruction-admission tests passed on Windows, Linux and macOS with Python 3.11 and 3.12. Each CI job also ran the demo and built both distributions. A separate clean non-editable wheel installation passed its console commands.

The portable gateway completed eight actual Strata model requests. All eight returned the exact expected integer, both request classes used learned timing on their fourth request, and the gateway drained. The separately operated backend shut down with all six retained process identities confirmed exited. This is a serial functional check, not a speed benchmark or qualification of every application.

Fresh public source reconstructions matched 88 Strata and seven Hermes changed files. The reconstructed code passed 54 Strata CPU tests and two Hermes intent tests.

Reproduce

Clone the tag, install requirements.lock and the package, then run adaptive-inference demo. See the quick start, integration guide, and evidence ledger. The wheel installs the gateway; clone or extract the source distribution for examples, citations and optional reconstruction scripts. SHA256SUMS.txt covers the attached wheel and source archive.

Known scope and next work

The generic gateway does not preempt GPU kernels, resize arbitrary native caches, own backend processes or grant tool authority. The other seven engine profiles and ten client recipes have contract/configuration evidence, not universal installed-application qualification.

The first real-backend attempt timed out after Strata idle-unloaded and its recorded reload footprint exceeded available RAM. That failure is retained in the ledger. The accepted run exercised a freshly ready backend; idle reload coordination remains future native-adapter work. Historical Windows owner software is not bundled in the source overlays.

Continue shared development here, then port focused capability changes upstream. The plan, 43-PR registry, and research attribution preserve that relationship.