Skip to content

kinfer v0.1.10

Choose a tag to compare

View all tags
@localkin localkin released this 04 Nov 17:04
· 33 commits to main since this release
v0.1.10
d9da3fe

kinfer launch starts Claude Code or Codex on a local model, and kinfer now answers the other requests those agents make, besides the ones for a reply.

curl -fsSL https://raw.githubusercontent.com/LocalKinAI/kinfer/main/scripts/install.sh | sh
kinfer launch claude -model qwen3.8                 # Claude Code
kinfer launch codex -model ornith-1.5:35b           # Codex
kinfer launch -host 192.168.0.21:11590 claude       # a kinfer on another machine; asks which model
kinfer launch claude -model qwen3.8 -- --continue   # the agent's own arguments go after --
  • kinfer launch. It starts the agent pointed at kinfer.
    • Where it connects: to the daemon kinfer run uses, started if nothing answers. With -host, it connects to a kinfer on another machine instead.
    • Menus: with no agent or no model given, it asks. It lists only the agents installed, leaves out embedding models, and puts the loaded model first.
    • Before the agent starts:
      • the model is loaded, so the first turn doesn't wait for it;
      • the agent is told how long one conversation can grow, so it summarizes its history before kinfer would have to drop turns;
      • Claude Code gets the model under every name it asks for one, so no background request asks for a claude-haiku the server doesn't have;
      • Codex gets a provider on its command line.
    • Nothing written: neither agent's configuration is changed, so there's nothing to restore afterwards.
  • /api/ps reports the context a loaded model gives each conversation (context_length, as Ollama's does) and the most one conversation can have (context_max).
  • The other requests agents make:
    • POST /v1/messages/count_tokens counts a request as /v1/messages renders it. On ornith-1.5:35b it gave 288 and 284 for requests whose usage.input_tokens came back 288 and 284.
    • GET /v1/models/{id} answers with both OpenAI's and Anthropic's fields.
    • HEAD /api/hello, Claude Code's first request in every session, now gets an answer. It used to count as a rejected request each time.
    • /v1/models now lists the models borrowed from Ollama's store as well as kinfer's own.
From a Mac mini, against ornith-1.5:35b on an M3 Ultra Claude Code 2.1.318 Codex 0.155.1
write a word counter and three unittest cases, run them, fix what fails 37 s, model load included; tests pass 15 s; tests pass
requests kinfer refused in a session none none

The patched libraries are unchanged (darwin_arm64_b11998k5). CHANGELOG.md

kinfer-v0.1.10-darwin-arm64.tar.gz sha256 690e05449e09cdca907e851216682b407c22afe09a3267731fc351a52f44f0d4

Assets 4
Loading