Skip to content

codeaf v0.8.3

Latest

Choose a tag to compare

@github-actions github-actions released this 19 Nov 22:21

Fixed

  • A slow local model gets ten times the wait, and response.patience changes it — #2854 · engine chat docs

    4 things that are no longer true
    • A model on your own machine or network (Ollama, LM Studio, llama.cpp, a box on the LAN or Tailscale) was held to the hosted figures: 2 minutes for response headers, 10 seconds of silence after headers, 45 seconds quiet mid-reply, a 5-minute reply wall, 150 seconds for a non-streamed reply, and a 2-minute compaction summary. Under the default response.patience: auto a local endpoint now gets ten times each of those, and the turn's give-up stretches with them; hosted endpoints keep the old figures.
    • A slow request to a local endpoint was sent again to the same server after 10 seconds, so one GPU did the work twice. Under auto a local endpoint is never sent a duplicate unless the gateway lists more than one provider for the model.
    • Nothing could make codeaf wait longer for a quiet model; response.attempts only scaled the give-up. response.patience (env CODEAF_RESPONSE_PATIENCE, the patience row on the Models tab) takes auto or a number from 1 to 100 that applies to every endpoint the same way, and 1 is exactly the behaviour before this change. A bad value is refused naming the row or the variable.
    • A local server that sent its headers at once and its first token after a long prompt read was cut at the action deadline (100 seconds under auto) and asked again, every time, so a first token later than that never arrived. Under auto a request to a local endpoint that lists no second provider is never cut to ask the same machine again; it waits for the first-token limit, 15 minutes on a conversation turn.

    A local model can spend minutes reading a long prompt before its first token,
    and it is the only machine there is to ask. "Local" is decided from the address
    alone: loopback, private, carrier-grade and link-local addresses, .local,
    .lan and .internal names, host.docker.internal, a single-word host, or the
    Ollama connection wherever it runs. A saved row reaches an open conversation on
    its next model request.

Install this stable build:

curl -fsSL https://raw.githubusercontent.com/Agent-Field/CodeAF/1a682dfa838ff56042b0499b48d13b46214e7157/scripts/install.sh | bash -s -- --stable
curl -fsSL https://raw.githubusercontent.com/Agent-Field/CodeAF/1a682dfa838ff56042b0499b48d13b46214e7157/scripts/install.sh | VERSION=v0.8.3 bash