Skip to content

Loki Mode v11.155.0

Choose a tag to compare

@github-actions github-actions released this 17 Nov 23:40
· 189 commits to main since this release
Immutable release. Only release title and notes can be modified.

Checking that finishes on a repository Loki has not seen

Why: on 2026-11-17 Loki was run on issues from microsoft/playwright, anomalyco/opencode and NousResearch/hermes-agent. In twelve finished runs it never labelled its work VERIFIED, and in most the change itself passed an independent check. The label was decided by Loki's own start-up and checking: about five minutes testing the whole repository before the model began, a verify step that tried every package and ran out of time, a test command that could not start counted as failed tests, and a start that failed with no branch checked out.

Measured without a model, on a five-package test repository: time before work begins 7.9 s to under 1 s; test suites run after a change to one package 5 to 3 (the changed package and the two that depend on it); a start with no branch checked out now completes; a relevant suite that cannot finish now ends PARTIAL and named, where it sealed VERIFIED before; a test command that cannot start ends PARTIAL with no fix round, where it ended FAILED. Loki's own test files on this build: run-safety 186 passed, 0 failed; scope-and-fix 856 passed, 0 failed; engine10-e2e-stub 177 passed, 0 failed; stacks 60 passed, 0 failed.

Not measured yet: a real run on a large repository with this build. The base comparison in a linked JavaScript workspace has only run on the small test repository and may refuse a very large install, in which case an already-failing suite blocks as before. A change that edits a dependency manifest gets no base comparison. One test file (decompose-and-intake) has 10 failing cases that predate this release and are being repaired. LOKI_E10_BASE_ON_DEMAND=0 restores the previous behaviour.

  • In a repository with several packages, Loki no longer runs every package's tests on the base commit before the model starts. Measured on a recorded run (opencode, 36 packages, 900 s limit): 308 of the first 311 seconds went to 4 of the 36 suites, and the other 32 were never measured. The base commit is now tested after the change exists, and only for the suites that fail on the change. On Loki's own five-package test workspace the start went from 7.9 s to 0.4 s. LOKI_E10_BASE_ON_DEMAND=0 restores the old behaviour, including every point below. A repository with one package behaves as before.
  • Verify runs the tests of the packages a change touched and of the packages whose manifest depends on them, not every package. The same recorded run tried 36 suites for a change to one package and ran out of time on the fourth. Packages not run are listed in the receipt under "not applicable" with the reason; a touched package whose tests did not run is listed under NOT PROVEN and the run is never VERIFIED. A change to a workspace-level manifest, lockfile or compiler, linter or test-runner config still runs everything.
  • Verify sizes itself from the time left. It may now use the time held back for a fix round (155 s became 335 s on the recorded run), each package suite gets what is left of the stage rather than a fixed 300 s, and a suite with no time left is listed as not run instead of the stage being cut. One verify is bounded at 15 minutes (LOKI_E10_VERIFY_MAX_S); a recorded run on a one-hour limit spent 21 minutes in one.
  • A test command that cannot start is no longer reported as failed tests. Recorded on hermes-agent: 33 test files each "failed" in 0.06 s with No module named pytest, a fix round was spent on them, and a correct one-line change was labelled FAILED. A check that fails with no test id is now run once more with the change removed; when it fails with the same lines there too, and no line names a changed file or symbol, it is listed as "the test command does not run in this checkout", is not given to a fix round, and the run is PARTIAL. Anything else stays a failure.
  • A project virtualenv is used to run pytest only when it can import pytest. In the hermes-agent run the .venv did not exist at the start; a uv run inside the session created one without pytest (the repository keeps pytest in a dependency group uv does not install by default), and every test file was then run on it. The tests now run on the interpreter outside the virtualenv, and the receipt says which interpreter ran them and why the virtualenv was skipped.
  • In an installed workspace (npm, yarn, bun or pnpm keep a link to each member package in node_modules), the run on the base commit reads the base copy of a sibling package. Loki refused to share such a node_modules with its base run at all, so an already-failing suite could not be compared and blocked the run.
  • A TypeScript check that is red only in files the change did not touch, with every diagnostic also present on the base commit, is disclosed and no longer blocks (it was opt-in as LOKI_E10_LINT_UNTOUCHED=1; =0 turns it off). One new diagnostic, a diagnostic in a touched file, or a changed compiler config still blocks.
  • loki start works from a detached HEAD checkout. It failed at intake in 3 seconds; it now starts from that commit and leaves the checkout detached on it at the end.