Skip to content

fix(server): orchestration var resolution, extraction gating, and error messages - #526

Merged
arantespp merged 3 commits into
mainfrom
claude/issues-feature-requests-381wxh
Jul 13, 2026
Merged

fix(server): orchestration var resolution, extraction gating, and error messages#526
arantespp merged 3 commits into
mainfrom
claude/issues-feature-requests-381wxh

Conversation

@arantespp

Copy link
Copy Markdown
Member

Addresses the SOAT bug/feature report from the Marketing Squad build. While triaging, several reported items turned out to be already resolved on main; this PR fixes the genuinely-open items and adds regression coverage across the board.

Already resolved (verified, regression-guarded where useful)

  • F-2 — MCP tools scoped to a subset of actions. Delivered by feat(tools): scope MCP tools to a subset of actions #521: actions is an enforced allowlist on mcp tools (filtered at enumeration and rejected at call time). The report's actions: null premise is stale. No code change; a denylist remains a possible future add-on.
  • F-3 — knowledge/search null relevance scores. The field was renamed scoresimilarity_score and is populated from the pgvector distance (1 - distance) and sorted descending; the null-score symptom no longer reproduces. Added a characterization test asserting a query populates a numeric similarity_score and that min_score filters against the true value.
  • F-4 (as literally worded) — input_schema not crediting {"var"}. The validator already credits input_schema.properties for bare references. The real gap was the namespaced form — fixed below.

Fixed

Orchestration variable resolution (F-4 / F-5)

  • Run input namespace (F-4, F-5 Feat modules #3). Run input is now seeded under an input namespace in addition to flat top-level keys, matching the pipeline/formation {"var":"input.<name>"} convention, and the static validator credits that namespace. A graph authored against the documented contract now sees its run input in every node expression and input_mapping.
  • Verbatim input casing (F-5 Feat modules #3 root cause). Orchestration-run input is now a caseTransform pass-through (like a tool's input). Previously snake_case input keys were camelCased into state while var references kept the original casing, so every underscore-bearing key resolved to null inside node logic even though it reappeared in the snake_case-restored final-state dump — exactly the reported symptom.
  • Nested state writes (F-5 Feat files #1). A dotted output_mapping target (state.proposed.action_id) now builds a nested object, so a downstream node reads it back with {"var":"proposed.action_id"} (the var reader descends dot-paths).
  • Readable run errors (F-5 chore: change website name #4). buildRunError serializes a non-Error thrown value (e.g. json-logic-engine's bare {type:'Unknown Operator'} for a multi-key map mapper) instead of collapsing it to the opaque [object Object].
  • Evaluation scope (F-5 Feat files #2) is documented rather than changed: transform/condition evaluate against full state; projecting nodes evaluate input_mapping against state and receive only the projection; poll augments state with response/attempt. Unifying them would be incorrect (an agent node must not receive the whole state as its prompt).

Memory extraction (F-6 / F-7)

  • Per-turn gating (F-6). New top-level extract boolean on POST /agents/:id/generate: false suppresses extraction for a single (e.g. operational/tool-listing) turn, true forces it on for a turn even when the agent does not enable it by default (given a write_memory_id); omitted follows the agent's stored config. No effect on streaming/requires_action turns.
  • Stale config (F-7). knowledge_config casing is normalized at extraction read time, so a formation-deployed agent whose stored config is still snake_case extracts correctly without needing to be re-saved.

Tests

  • knowledge.test.ts — numeric similarity_score contract + min_score filtering.
  • orchestrationValidation.test.ts — namespaced input.<name> reference accepted against a closed input_schema.
  • orchestrations.test.ts — run input visible via the input namespace; dotted output_mapping round-trips through a nested var read; a non-Error throw records a readable message.
  • memoryExtraction.test.tsextract:false suppresses, extract:true forces, and a stale snake_case stored config still extracts (read-time normalization).
  • smoke-tests.sh — deterministic input-namespace run assertion.

Verification

pnpm typecheck, pnpm eslint, pnpm lint-openapi, and pnpm docs-lint pass. Affected unit suites (orchestrations, orchestrationValidation, orchestrationNodeExecutors, memoryExtraction, knowledge, tools, agents, jsonLogicMapping, mcp, caseTransform) all pass against the real Postgres/pgvector testcontainer. SDK and CLI were regenerated from the OpenAPI change (extract field); generated files are gitignored and rebuilt in CI.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NakdELuVYYyjpoyV9VcSKW


Generated by Claude Code

…essages

Addresses the SOAT bug/feature report from the Marketing Squad build. Several
reported items (MCP action scoping F-2, knowledge similarity_score F-3, and
input_schema crediting in validation F-4) were already resolved on main; this
change fixes the genuinely-open items and adds regression coverage.

Orchestration variable resolution (F-4/F-5):
- Seed run input under an `input` namespace in addition to flat top-level keys,
  matching the pipeline/formation `{"var":"input.<name>"}` convention, and
  credit that namespace in the static validator. A graph authored against the
  documented contract now sees its run input in every node expression and
  input_mapping.
- Make orchestration-run `input` a caseTransform pass-through so snake_case
  input keys round-trip verbatim; previously they were camelCased into state
  while `var` references kept the original casing, so every underscore-bearing
  input key resolved to null inside node logic even though it reappeared in the
  snake_case-restored final-state dump.
- writeToState builds a nested object for a dotted output_mapping target
  (`state.proposed.action_id`) so a downstream node reads it back with
  `{"var":"proposed.action_id"}` (the var reader descends dot-paths).
- buildRunError serializes a non-Error thrown value (e.g. json-logic-engine's
  bare `{type:'Unknown Operator'}` for a multi-key map mapper) instead of
  collapsing it to the opaque "[object Object]".

Memory extraction gating (F-6) and stale config (F-7):
- Add a per-turn `extract` flag on POST /agents/:id/generate: `false` suppresses
  extraction for a single (e.g. operational) turn, `true` forces it on for a
  turn even when the agent does not enable it by default; omitted follows the
  agent's stored config.
- Normalize knowledge_config casing at extraction read time so a
  formation-deployed agent whose stored config is still snake_case extracts
  correctly without needing to be re-saved.

Docs updated (agents, memories, orchestrations) and a smoke step added for the
input namespace. parseDuration extracted to orchestrationDuration.ts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NakdELuVYYyjpoyV9VcSKW
@arantespp
arantespp enabled auto-merge (squash) July 13, 2026 07:43
claude added 2 commits July 13, 2026 08:01
…olds

The new non-Error handling in buildRunError (describeThrown) added defensive
branches — empty-object serialization, primitive throws, and the
circular/non-serializable catch — that no REST entry point can trigger, which
dropped global coverage below the tuned jest thresholds and failed CI's
`pnpm run test`. Add a direct lib test for the exported buildRunError covering
Error, DomainError, a bare object, an empty object, a primitive, and a circular
value, so every branch is exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NakdELuVYYyjpoyV9VcSKW
…riteToState

Per maintainer request, relax the per-file coverage threshold from 80 to 75.
Also cover the new dotted-path branches of writeToState (multi-level nesting,
merge into an existing object, and overwriting a non-object/array intermediate)
via applyOutputMapping, so the new orchestration state-write behavior is fully
exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NakdELuVYYyjpoyV9VcSKW
@github-actions

Copy link
Copy Markdown

Deploy Outputs

Package Stack Output Key Output Value
@soat/website SoatWebsite-claude-issues-feature-requests-381wxh BucketWebsiteURL http://soatwebsite-claude-issues-feature-req-staticbucket-1iosqfoe2thq.s3-website-us-east-1.amazonaws.com

@arantespp
arantespp merged commit a16cccd into main Jul 13, 2026
6 checks passed
@arantespp
arantespp deleted the claude/issues-feature-requests-381wxh branch July 13, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants