Skip to content

chore(tests): Avoid network, sleep and more during tests - #11050

Open
richiejp wants to merge 12 commits into
mudler:masterfrom
richiejp:perf/test-suite-10m
Open

chore(tests): Avoid network, sleep and more during tests#11050
richiejp wants to merge 12 commits into
mudler:masterfrom
richiejp:perf/test-suite-10m

Conversation

@richiejp

@richiejp richiejp commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator

Description

This PR fixes #11048

Notes for Reviewers

  • feat(llama-cpp): route Score through the slot loop
  • feat(realtime): classifier wire types and pipeline config
  • feat(realtime): classifier response flow
  • fix(realtime): bound the VAD tick's scan window and buffer retention
  • fix(backend): let per-model threads override the global default
  • chore(gallery): single-thread the silero VAD
  • docs(realtime): classifier mode, VAD scan window, threads precedence
  • perf(llama-cpp): score all candidates in one batched decode
  • fix(realtime): gate scoring capacity by model usecase
  • fix(ci): honor APT mirrors in the prebuilt llama-cpp compile step
  • feat(realtime): classifier argument slots via constrained completion
  • feat(realtime): splice filled slot values into classifier replies
  • fix(realtime): harden classifier slot completion
  • feat(realtime): prewarm the classifier scoring prompt on registration
  • perf(llama-cpp): checkpoint scoring at the caller-declared stable prefix
  • fix(realtime): align classifier cache guidance
  • fix(llama-cpp): guard score task for fork backends
  • fix(dev): generate gRPC code before commit lint
  • test: make coverage failures observable
  • test: parallelize coverage without remote fixtures
  • test: add offline resource infrastructure
  • test: enforce offline resource replay
  • test: harden offline resource refresh
  • test: expose slow coverage waits
  • test: eliminate avoidable wall-clock waits
  • test: remove repeated fixture startup waits

Signed commits

  • Yes, I signed my commits.
  • Documentation updated (docs/content/) for user-facing changes, or not applicable

@richiejp
richiejp force-pushed the perf/test-suite-10m branch from ec4ad92 to e9b8d2d Compare July 23, 2026 08:56
@richiejp
richiejp force-pushed the perf/test-suite-10m branch from 3e38852 to ec33627 Compare July 29, 2026 13:01
richiejp and others added 12 commits August 3, 2026 09:53
@richiejp
test: make coverage failures observable
4de742b 
Keep per-root logs, reject concurrent coverage runs, and avoid relying on /bin/sleep in the worker timeout test.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: parallelize coverage without remote fixtures
dbe7d8f 
Assisted-by: Codex:gpt-5 [apply_patch] [exec_command]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: add offline resource infrastructure
f1baae2 
Introduce versioned resource manifests, a checksum-verified CAS preparer, offline test wrappers, and a guarded network transport. Replace live Hugging Face, GitHub, and OCI cases with deterministic fixtures and inject fixture metadata into importer discovery.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: enforce offline resource replay
dd9a2ce 
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: harden offline resource refresh
c4eab25 
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: expose slow coverage waits
bfe76b2 
Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: eliminate avoidable wall-clock waits
ae191c0 
Inject a clock into Hugging Face retry handling, reuse a process-scoped PostgreSQL container with per-spec schemas in the nodes suite, and poll local import jobs promptly.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: remove repeated fixture startup waits
4c301fb 
Share PostgreSQL fixtures across parallel endpoint and agent suite workers, and make the worker Free deadline injectable so the wedged-backend test does not spend five seconds on wall-clock time.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: fix offline resource CI portability
5415ec9 
Normalize Docker archive metadata before content addressing, derive archive checksums during explicit refreshes, make network lint portable to macOS, and prepare distributed images before running their offline suite.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
ci: cache Go modules before offline tests
35dcbe0 
Warm the complete module graph before the Linux and macOS test jobs enter offline replay mode, so tool dependencies such as Ginkgo are not fetched through the guarded proxy.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
test: drop the static network lint in favour of real isolation
8f1f679 
The offline test suite already prevents tests from reaching the network
twice over: run-test-linux-offline.sh puts the test process in a cgroup
and REJECTs egress outside the private ranges, and HardenedTransport
installs testnetwork.LocalGuard to refuse dials that resolve to a public
address. Both fail the test with a precise error at the moment of the
dial.

test-network-lint.sh added neither. Its diff stage defaulted to a HEAD
base, so on a clean checkout it compared the tree against itself and
inspected nothing; the branch's own commits were never examined. It only
produced output when an earlier job step dirtied the tree, and then it
matched a bare https?:// against whatever changed. make react-ui runs
npm install rather than npm ci, so CI rewrote
core/http/react-ui/package-lock.json and the lint reported an npm
registry URL as forbidden test network access:

  +      "resolved": "https://registry.npmjs.org/hono/-/hono-4.12.25.tgz",

Its fingerprint stage was self-defeating in a quieter way: hashing the
whole tree's network-mechanism inventory meant every rebase onto a master
that touched any _test.go needed a manual baseline bump, so the check
mostly caught its own staleness.

Remove the script, its make target and the two prerequisite edges, along
with the test-network: fixture markers that existed only to suppress it.
The isolation itself is untouched.

Assisted-by: Claude:claude-opus-5 [go vet]
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@localai-org-maint-bot @richiejp
ci: keep hidden files in the offline test bundle artifact
8a54235 
Cherry-picked from 15a37b0 on the remote branch. The offline bundle lives
under .cache/, which actions/upload-artifact skips by default, so the Linux
job packed an artifact missing the very file the next step restores.

The other half of 15a37b0 moved test-network-lint out of the `test` and
`test-coverage` prerequisite lists into a recipe line, so parallel make could
not fingerprint the tree while generated fixtures were still changing. That
is dropped: the preceding commit removes the lint entirely, and the race it
worked around is one more reason a whole-tree fingerprint was the wrong
mechanism.

Assisted-by: Codex:gpt-5
Signed-off-by: Richard Palethorpe <io@richiejp.com>
@richiejp
richiejp force-pushed the perf/test-suite-10m branch from 15a37b0 to 8a54235 Compare August 3, 2026 10:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

LocalAI QA: Make tests go fast

2 participants

@richiejp @localai-org-maint-bot