release: promote testnet-canary to main for 10.0.11 - #1999
Conversation
* Fix release CI test expectations * fix(epcis): fail fast on invalid payloads * Merge pull request #1984 from OriginTrail/release/10.0.10-bump chore(release): bump version set to 10.0.10 --------- Co-authored-by: Viktor Pelle <vikpelle@gmail.com>
The 10.0.10 sync (#1986) was squash-merged, which flattened it into a single commit parented on 8334c0a — a pre-release commit on main. The content came across correctly (main and testnet-canary trees were already byte-identical), but git no longer saw main's release commits as ancestors of testnet-canary, so `git rev-list testnet-canary..main` reported four phantom commits: f61f335 Merge pull request #1984 (release/10.0.10-bump) f516586 Merge pull request #1983 (release/10.0.10) e89f8f6 fix(epcis): fail fast on invalid payloads 0112e78 Fix release CI test expectations This is a real merge commit, so the histories join and those stop being reported as missing. It changes no files. Squash is right for feature PRs; for a branch-sync PR it defeats the purpose, because the ancestry is the payload. Future main -> testnet-canary syncs should use 'Create a merge commit'. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
fix: prevent scoped API queries from starving store work
chore: sync main into testnet-canary after CI parity (#1992)
fix(integrations): conform the CLI to the published registry schema
…lowup fix: address query scheduler follow-up review feedback
| const promise = this.discoverScopedContentGraphAllowList( | ||
| contextGraphId, | ||
| subGraphName, | ||
| sharedOptions, |
There was a problem hiding this comment.
🔴 Bug: Shared graph discovery drops API cancellation and can leave orphan store work
What's wrong
The route promises that disconnected API callers cancel queued/in-flight store work, but this shared discovery path strips the signal before issuing the actual store reads. That means the expensive partition discovery can keep consuming scheduler capacity after the HTTP caller is gone.
Example
A client sends /api/query with includeContextGraphPartitions: true and a GRAPH ?g query, then disconnects while the allow-list discovery is running. The caller gets cancelled through raceAgainstCallerAbort, but the underlying discovery store reads continue because they were started without the caller signal.
Suggested direction
Use a shared-flight owner that tracks active subscribers and aborts the store discovery when the last subscriber cancels, or avoid sharing flights for caller-cancellable API reads.
For Agents
Look in packages/query/src/dkg-query-engine.ts around sharedDiscoveryStoreOptions and resolveScopedContentGraphAllowList. Preserve shared-flight behavior for concurrent callers, but ensure a single disconnected caller does not leave discovery work running. Add a test where the only subscriber aborts and the discovery store query observes the abort, plus a second test proving one aborted subscriber does not cancel another active subscriber.
| error_message: string; | ||
| }): void { | ||
| this.stmt('cancelOp', ` | ||
| UPDATE operations SET status = 'cancelled', duration_ms = @duration_ms, |
There was a problem hiding this comment.
🟡 Issue: Cancelled operations skew dashboard success-rate metrics
What's wrong
The PR adds a non-error terminal status for caller disconnects, but the existing metrics treat every row in operations as part of the denominator. Cancellations therefore lower success rates without increasing error counts, producing inconsistent operational health data.
Example
With one successful operation and one disconnected /api/query, getOperationStats() reports totalCount = 2, successCount = 1, errorCount = 0, and successRate = 0.5. The cancellation is not an error, but it still depresses the success rate as if it were a failed/incomplete operation.
Suggested direction
Update stats queries to either exclude terminal cancellations from success-rate denominators or expose them as a separate count and compute rates over completed success/error outcomes only.
Confidence note
This assumes the dashboard success-rate metrics are meant to exclude caller cancellations from failure-like outcomes, which matches the new tracker test wording but should be confirmed with product expectations.
For Agents
Look in packages/node-ui/src/db.ts at the new cancelled status and the operation stats queries. Decide whether cancelled operations should be excluded from rate denominators or surfaced as cancelledCount; add a DB-level test for one success plus one cancelled operation.
| return Array.isArray(v) && v.every((x) => typeof x === 'string'); | ||
| } | ||
|
|
||
| // INVARIANT: this must never be STRICTER than the registry's published JSON |
There was a problem hiding this comment.
🟡 Issue: Do not keep hand-encoding the registry schema
What's wrong
This change fixes several schema drift cases by adding more manual schema logic. That makes the immediate behavior better, but structurally it doubles down on the design that caused the drift: the CLI, fixture copy, and registry schema now all describe the same contract independently.
Example
A future registry addition such as a new service runtime would need coordinated edits in the published schema, the vendored fixture, the hand-written switch, and the tests. Missing any one of those brings back the exact drift this PR is trying to prevent.
Suggested direction
Use the registry schema, or generate typed validators from it, instead of expanding a parallel hand-written validator plus fixture copy. The code-judo move is to delete the drift-prone mirror rather than add more tests around it.
Confidence note
This assumes the CLI can either consume the registry schema at runtime or generate a parser from it at build time; if runtime dependency size is a constraint, generation still addresses the same structure problem.
For Agents
Look at packages/cli/src/integrations/schema.ts and the new registry fixtures. Preserve the current lenient-read/installability split, but make the published schema or a generated projection the canonical source of readable shapes; keep installability checks such as resolveNpmGlobalService separate. Prove with the existing registry contract cases.
There was a problem hiding this comment.
🟡 Issue: The registry parser still duplicates the schema it says must be canonical
What's wrong
The PR documents previous schema drift and then adds more hand-maintained schema knowledge to the same validator. That leaves the structural problem intact: every schema evolution still requires synchronized edits across interfaces, the type guard, install dispatch, installed detection, and fixtures.
Example
If the registry adds a new valid install kind, this parser still needs a TypeScript union update, isValidInstallSpec branch, command handling, and tests before the CLI can even read the entry. That is exactly the drift this PR is trying to prevent.
Suggested direction
Use the published schema, or a generated validator/model, as the canonical readability boundary instead of expanding the manual switch. If leniency for future fields is required, model that explicitly with an UnknownInstallSpec rather than encoding compatibility as comments around a closed union.
For Agents
Rework packages/cli/src/integrations/schema.ts so the registry schema, or a generated/derived validator from it, owns readability. Represent unknown-but-readable install specs explicitly, then let installer/dispatcher code decide whether a known kind is automatable. Keep behavior for existing known kinds the same.
| * that as "not installed": it would tell a user an integration is missing when | ||
| * the truth is that their config could not be read. | ||
| */ | ||
| export function readRegisteredServerKeys(target: ClientTarget): ServerKeyProbe { |
There was a problem hiding this comment.
🟡 Issue: Extract MCP config parsing instead of expanding mcp-setup.ts
What's wrong
The PR turns a very large command/setup module into a shared utility boundary by exporting internal config parsing concepts. That makes the module less cohesive and gives unrelated integration code a dependency on a file that also owns interactive setup, serialization, and command behavior.
Example
detectInstalled now imports detectClients, readRegisteredServerKeys, and config-shape types from ../mcp-setup.js; a later change to setup flow or client target representation can now ripple into integration detection even though detection only needs a read-only config parser.
Suggested direction
Move shared MCP client config discovery/parsing out of the setup command file. The setup command should orchestrate writes; integration detection should consume a small read-only parser API owned by a neutral module.
For Agents
Extract the MCP client target model plus config read/parse helpers into a focused module such as mcp-client-config.ts. Have both mcp-setup.ts and integrations/detect-installed.ts import that module. Preserve existing setup behavior and the new registered-server-key tests.
| } | ||
|
|
||
| /** Map retryable, pre-dispatch read shedding without changing write routes. */ | ||
| export function respondIfApiQueryStoreBusy(res: ServerResponse, err: unknown): boolean { |
There was a problem hiding this comment.
🟡 Issue: Use a typed scheduler-busy boundary instead of a local magic string
What's wrong
The route now owns a copy of a storage-layer error code and reaches into an unknown object shape to decide HTTP semantics. That is a brittle boundary: the producer and consumer of the scheduler-busy contract can drift without type pressure.
Example
If the storage scheduler later renames the code or adds a typed retry hint, this route will still compile while silently mapping by stale string/object shape.
Suggested direction
Move the scheduler-busy detection contract to the storage package, or import a typed predicate/error constant from there. Keep the route responsible for HTTP serialization, not for reconstructing storage error shape by hand.
For Agents
Look at packages/storage/src/store-priority-scheduler.ts and packages/cli/src/daemon/routes/query.ts. Export a stable constant or predicate such as isStoreSchedulerBusyError, and use that in the route response mapper while preserving the 503/Retry-After response.
| return -1; | ||
| } | ||
|
|
||
| function skipSparqlIriRef(sparql: string, start: number): number | null { |
There was a problem hiding this comment.
🟡 Issue: Avoid introducing a second SPARQL scanner
What's wrong
The extraction improves locality for the new VALUES logic, but it duplicates low-level parser rules that already exist in the query engine. Parser duplication is expensive debt because subtle lexical behavior has to stay identical across every graph-safety rewrite.
Example
A future fix to SPARQL IRIREF scanning or keyword-boundary handling would need to be applied in both files. If only one copy changes, callerGraphValuesAreAuthorized and the existing graph-scope rewriters can parse the same query differently.
Suggested direction
Centralize the reusable SPARQL token-scanning primitives instead of cloning them into the new graph-scope helper module.
For Agents
Move the shared lexical scanner helpers into packages/query/src/sparql-utils.ts next to skipSparqlStringLiteral, then import them from both dkg-query-engine.ts and sparql-graph-scope.ts. Preserve the new VALUES elision behavior and existing graph-scoping tests.
| }); | ||
| }); | ||
|
|
||
| // ── Registry ↔ CLI contract ─────────────────────────────────────────────── |
There was a problem hiding this comment.
🟡 Issue: Split this test file before it crosses 1k lines
What's wrong
This PR pushes an existing 614-line file past the 1k-line threshold. The added suites are valuable, but they are not one cohesive test subject; keeping them together makes future failures and fixture setup harder to navigate.
Example
A reader looking for the existing installMcp tests now has to scan through registry schema compatibility, command-runner fixtures, process-exit spies, and install-dispatch coverage in the same file.
Suggested direction
Decompose the appended suites by responsibility instead of letting this general integration test file become the catch-all for registry, installer, and Commander behavior.
For Agents
Split the new suites into focused files, for example integrations-registry-contract.test.ts, integrations-mcp-install.test.ts, and integrations-commands.test.ts. Extract shared registry/server fixtures into a small test helper if needed. Preserve the same assertions.
There was a problem hiding this comment.
🟡 Issue: This PR pushes integrations.test.ts past 1k lines instead of decomposing it
What's wrong
The added suites take an already broad test file from under the 1k-line threshold to well over it. That is a maintainability regression even though the tests are useful: this file is now a grab bag of registry client, installer, schema-contract, and command-dispatch coverage.
Example
A future change to installMcp or registry parsing now has to scan one 1k+ integration mega-suite containing registry-client tests, installer tests, registry-schema contract tests, and command-wiring tests.
Suggested direction
Decompose the new registry-contract and Commander tests before merging. Keeping each behavioral area in its own file will make failures and future edits much easier to localize.
For Agents
Split the newly added suites into focused files, for example integrations-registry-contract.test.ts and integrations-commands.test.ts. Preserve the current assertions and fixtures; only move shared test fixtures/helpers into a small test helper module if needed. The existing tests should pass unchanged after the split.
| assertionName, | ||
| subGraphName, | ||
| callerAgentAddress, | ||
| signal: queryLifecycle.signal, |
There was a problem hiding this comment.
🟡 Issue: The new /api/query lifecycle wiring is not verified at the route boundary
What's wrong
The changed user-facing behavior depends on handleQueryRoutes actually threading the lifecycle into agent.query and mapping scheduler shedding in its catch block. Current coverage verifies the pieces in isolation, so the public endpoint could regress while the new tests still pass.
Example
A regression that removes signal: queryLifecycle.signal or priority: queryLifecycle.priority from the agent.query call would still leave the helper tests and lower-layer forwarding tests green, while /api/query would stop using the background lane or stop cancelling disconnected reads.
Suggested direction
Cover the actual route call site, not only the helper and downstream layers, so missing or miswired lifecycle options fail a test.
For Agents
Add a route-boundary test around handleQueryRoutes in the query route tests. Use a RequestContext with an agent query stub that captures options, then assert /api/query supplies signal, default background priority, and source: 'api.query'; also assert a StoreSchedulerBusyError thrown by that call becomes HTTP 503 with Retry-After. Keep lower-layer forwarding tests as they are.
There was a problem hiding this comment.
🟡 Issue: Route-level priority override is not verified
What's wrong
The PR documents DKG_API_QUERY_PRIORITY=normal as a canary rollback path, but the tests do not prove the actual /api/query route honors that override. They only prove the helper can resolve it. That leaves the public operational contract vulnerable to a hardcoded route value while tests stay green.
Example
A regression that changed the route call to priority: 'background' would still pass the current lifecycle unit test for DKG_API_QUERY_PRIORITY=normal, because that test never drives /api/query through handleQueryRoutes with the override enabled.
Suggested direction
Add a /api/query handler test that proves the configured normal lane reaches the real route handoff, not just the lifecycle helper.
For Agents
In packages/cli/test/query-route-lifecycle.test.ts, add a route-level case that calls configureApiQueryPriority('normal', ...), invokes handleQueryRoutes, and asserts the stubbed agent.query receives priority: 'normal' plus source: 'api.query'. Preserve the existing default-background route test as the control.
There was a problem hiding this comment.
🟡 Issue: GenUI scheduler-busy response path has no route test
What's wrong
The new behavior changes a user-facing route from a generic failure to a retryable 503 when store admission sheds the read. The only tests cover the shared helper and /api/query; they do not prove /api/genui/render is wired to that helper.
Example
Failing-test sketch: call handleQueryRoutes with path: '/api/genui/render', a valid body, configured llm.apiKey, and an agent.query stub that throws new StoreSchedulerBusyError(...); assert status 503, Retry-After: 1, and code: 'STORE_SCHEDULER_BUSY'.
Suggested direction
Cover the GenUI route branch directly so a future removal or misplacement of the helper call cannot regress to a 500 while helper-level tests still pass.
For Agents
Add a focused route test around the /api/genui/render entity-triple fetch branch in packages/cli/test/query-route-lifecycle.test.ts or the existing route test file. Keep the agent stub throwing the real StoreSchedulerBusyError, and assert the HTTP response contract rather than only the helper return value.
| getApiQueryPriority, | ||
| type ApiQueryPriority, | ||
| } from '../api-query-priority.js'; | ||
| export { |
There was a problem hiding this comment.
🟡 Issue: The query route widens its public surface just to expose startup-priority helpers
What's wrong
The route now exports unrelated daemon configuration helpers, which couples route consumers and tests to a module that should own HTTP behavior. This makes the module boundary less clear and normalizes using route files as convenience barrels.
Example
query-route-lifecycle.test.ts imports priority configuration through ../src/daemon/routes/query.js, making the HTTP route module a test-facing barrel for daemon startup configuration.
Suggested direction
Keep daemon priority configuration in api-query-priority.ts as the canonical module and avoid re-exporting it from the route. If the lifecycle helpers need a testable boundary, extract them into their own route-adjacent module rather than making routes/query.ts a mixed API surface.
Confidence note
This is about module boundary cleanliness; the behavior can remain exactly as-is.
For Agents
Update tests to import priority helpers from packages/cli/src/daemon/api-query-priority.ts directly. Keep routes/query.ts focused on request routing; if lifecycle helpers need direct tests, move them to a small api-query-lifecycle.ts module used by both the route and tests.
| <div style={{ display: 'flex', gap: 4, marginBottom: 3, flexWrap: 'wrap' }}> | ||
| {phases.map((p: any, i: number) => { | ||
| const color = p.status === 'error' ? '#ef4444' : PHASE_COLORS[p.phase] ?? PHASE_FALLBACK_COLOR; | ||
| const color = p.status === 'error' |
There was a problem hiding this comment.
🟡 Issue: cancelled is bolted on as repeated string branches instead of a status model
What's wrong
The PR adds a new operation status by scattering literal checks across the UI and DB aggregation code. That works locally, but it makes operation status semantics harder to reason about because color, badge class, terminal-state display, and health-rate inclusion are now separate ad-hoc branches.
Example
Adding another terminal status would require hunting through DB SQL, badge classes, Gantt colors, legends, tooltips, and detail rows. Missing one branch would silently render or count that status inconsistently.
Suggested direction
Centralize status metadata and health-rate policy so the new status is represented once. The UI components should ask for status color/label/class from a helper, and the DB stats code should share a named health-denominator policy instead of repeating success/error literals.
For Agents
Introduce a shared operation status model in node-ui, for example OperationStatus, TERMINAL_HEALTH_STATUSES, and operationStatusMeta(status). Use it for badge classes, phase colors/labels, and health denominator helpers or SQL fragments. Preserve the current success/error/cancelled/in_progress behavior.
3 participants
Summary
Promotes the
testnet-canarypayload tomainas the 10.0.11 release candidate. Frozen at canary tipe568ebe45.8 commits · 39 files · +5,069 / −676.
mainis fully contained intestnet-canary(merge-base ==maintip, 0 commits ahead), so this is a clean promotion with no conflict resolution.No Solidity, ABI, or deployment-registry changes — no on-chain deployment required.
No dashboard schema change —
SCHEMA_VERSIONstays at 31, so no migration and no rollback question.Contents
Three substantive PRs, all merged to
testnet-canary:fix: prevent scoped API queries from starving store work(closes mainnet: rs.loop.tick-threw from Store scheduler queue wait timeout in blazegraph.query #1989)fix: address query scheduler follow-up review feedback(follow-up to fix: prevent scoped API queries from starving store work #1991, two review rounds)fix(integrations): conform the CLI to the published registry schemaThe remaining commits are the #1986/#1993 syncs from main, a history-rejoin merge, and the review fixup
c50b39235.Why #1994 belongs here
#1991 landed the scoped-query starvation fix; #1994 is its review follow-up and touches
cli,node-ui,queryandstorage(+1,077 / −579), including a net −175 rework ofpackages/query/src/dkg-query-engine.tsand a newpackages/cli/src/daemon/api-query-priority.ts.The two load-bearing invariants were already correct in #1991 as merged, and remain so:
packages/cli/src/daemon/routes/query.ts:358returns'background'unless a caller explicitly requests'normal'.sharedDiscoveryStoreOptions()(packages/query/src/dkg-query-engine.ts:86-95) passes onlypriorityandsource, withraceAgainstCallerAbort()letting an individual caller bail without killing the shared flight.#1994 adds the robustness layer on top: one
QueryStoreReadContextbuilt at query entry so option threading cannot drift, cancelled operations preserved in the dashboard health rate (success / (success + error)), class-based rather than string-matchedStoreSchedulerBusyErrormapping, and shared SPARQL scanner primitives.CI status
ci.ymlnow includestestnet-canaryin both thepushandpull_requestbranch filters (#1992, canary parity), closing the gap that left 10.0.10's 497 commits unseen by the vitest lane until the promotion PR. This payload is therefore validated on canary before promotion, and this PR is a confirmation run.One item for reviewer awareness: #1994 was merged with two red checks —
CI gateandKosava: adapters + utilities + demo. The underlying failure isH-AC-31: re-run after replacement does not take a second backup (first-wins capture)inpackages/adapter-hermes, which timed out at 5,029ms against a 5,000ms limit. #1994 does not touchpackages/adapter-hermesat all (onlycli,node-ui,query,storage), so this reads as a flaky timeout rather than a regression. The push-triggered run on the merged tipe568ebe45should confirm; do not merge this PR until that run concludes green.Reviewer attention
Test Plan
e568ebe45(confirms theadapter-hermestimeout was flaky)maincommit (RELEASE_PROCESS.md §4) — after mergeFollow-up (separate PR, before tagging)
Version bump to 10.0.11 across all 21 manifests plus the
## [10.0.11]CHANGELOG section.[Unreleased]is currently empty.Not in this release: #1985 (node:sqlite runtime-floor enforcement) and #1971 (curated publish authz, which carries a
_VERSIONbump and needs a KAL redeploy) both remain open and correctly excluded from a patch release.🤖 Generated with Claude Code