Skip to content

test(e2e): add background task golden paths - #1748

Closed
ruiconti wants to merge 6 commits into
tasks/09-background-section-finalizationfrom
tasks/10-evals
Closed

test(e2e): add background task golden paths#1748
ruiconti wants to merge 6 commits into
tasks/09-background-section-finalizationfrom
tasks/10-evals

Conversation

@ruiconti

@ruiconti ruiconti commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Continuation of #1724 (closed to change base; content identical).

What

  • Add a deterministic fixture-tasks entry with experimental.tasks enabled.
  • Cover the background-task golden paths: one-step fanout of ten tasks, continued parent interaction while those tasks run, and lifecycle updates from every task back to the parent session.
  • Keep regression coverage for task HITL routing, concurrent session-create idempotency, input-batch ordering, and child-task exclusivity.
  • Run the fixture with an authored mockModel in every suite so the same orchestration executes against each workflow world without provider planning variance.

Golden paths

Path End-to-end assertion
Many-task fanout One model step starts ten background child sessions and returns ten distinct task receipts.
Parent interactivity All ten tasks first block on unreleased child approvals; the parent then accepts a normal follow-up and returns an ordinary tool-free reply.
Fanout parent updates After release, every fanout task completes independently and its completed-task notification is observed on the parent session.

Regression properties

Property End-to-end assertion
Task HITL routing A child approval surfaces on the parent session; its answer reaches the child without a parent model step.
Session-create idempotency Concurrent and serial /eve/v1/session retries resolve one session identity for one principal, while the same operation id under another principal creates a different session.
Task input-batch ordering Replaying Q1 after Q2 becomes current cannot clear Q2; the exact Q2 response still resumes the child and completes the task.
Child-task exclusivity Two same-batch task_send calls to one persistent child produce exactly one working receipt and one AGENT_BUSY result.

Shape

The fanout workers block on approval rather than a timer. The eval records every queued parent turn, proves all ten tasks are nonterminal before interaction, releases them explicitly, then matches updates by task id against the parent stream. Coalesced turns cannot hide updates or the interactive message.

Task completion is polled with task_peek; this fixture has no dependency on the removed task-wait behavior.

Stacked on #1747 (tasks/09). The full fixture passes locally with 27 gates.

Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
Signed-off-by: Rui Conti <ruiconti@gmail.com>
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
eve-docs Ready Ready Preview Aug 7, 2026 12:11am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
eve-docs-4759 Skipped Skipped Aug 7, 2026 12:11am

@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Bundle + Package Summary: apps/fixtures/weather-agent

Key takeaways

  • No notable deltas vs tasks/09-background-section-finalization (11528be).

Delta vs tasks/09-background-section-finalization (11528be)

Area Metric Baseline Current Delta
Package Packed tarball 7.69 MB 7.69 MB +664 B ⚠️
Package Unpacked publish size 28.79 MB 28.79 MB 0 B ➖
Package Installed footprint 72.18 MB 72.18 MB 0 B ➖
Package Published files 3072 3072 0
Package Installed files 6780 6780 0
Runtime Unique function payloads 2 2 0
Runtime Total function bytes 17.25 MB 17.25 MB -40 B ✅
Runtime Public routes 13 13 0
Changed function payloads vs tasks/09-background-section-finalization (11528be) (2)
Function Status Baseline Current Delta Route changes
functions/__server.func changed 8.62 MB 8.62 MB -20 B ✅ none
functions/.well-known/workflow/v1/flow.func changed 8.62 MB 8.62 MB -20 B ✅ none

eve init install

Metric Baseline Current Delta
Installed footprint 110.59 MB 110.59 MB 0 B ➖
Installed packages 121 121 0
dependencies 4 4 0
devDependencies 2 2 0
Dependency package bytes 43.26 MB 43.26 MB 0 B ➖
devDependency package bytes 5.04 MB 5.04 MB 0 B ➖
Build Metadata
  • Preset: vercel
  • Nitro: nitro@3.0.260610-beta
  • Output directory: apps/fixtures/weather-agent/.vercel/output
  • Build metadata timestamp: 2026-08-07T00:12:05.019Z
  • Route aliases: 13 public, 1 internal (14 total aliases)
  • Vercel routes in config: 16
  • Severity legend: 🔴 dominant/large, 🟠 notable, 🟡 watch, ⚪ small
Package Drill-Down

Package Details

  • Package: eve@0.30.7
  • Package directory: packages/eve
  • Tarball: 7.69 MB (eve-0.30.7.tgz)
  • Unpacked payload: 28.79 MB across 3072 published files
  • Installed footprint: 72.18 MB across 6780 installed files
  • Installed root package: 27.44 MB
  • Installed dependencies: 44.74 MB
  • Runtime dependencies: 2
  • Peer dependencies: 5 (4 optional)

Installed footprint is measured from an isolated temporary npm install of the packed tarball.

Heavy installed dependencies

  • eve: 27.44 MB (38.0%)
  • @rolldown/binding-linux-x64-gnu: 19.35 MB (26.8%)
  • ai: 6.65 MB (9.2%)
  • zod: 5.07 MB (7.0%)
  • undici: 3.50 MB (4.9%)
Publish payload breakdown
Published file size
🔴 dist/src/compiled/shadcn-registry/index.js       [################........] 9.68 MB 33.6%
🟠 dist/src/compiled/@photon-ai/chat-adapter-ime... [####....................] 2.42 MB 8.4%
🟠 dist/src/compiled/experimental-ai-sdk-code-mo... [###.....................] 1.51 MB 5.2%
🟡 dist/src/compiled/_chunks/node/undici-DWL_MYm... [#.......................] 502.4 kB 1.7%
🟡 dist/src/compiled/_chunks/workflow/undici-DWL... [#.......................] 502.4 kB 1.7%
🔴 Other published files                            [########################] 14.18 MB 49.2%
Installed footprint breakdown
Installed package size
🔴 eve                             [########################] 27.44 MB 38.0%
🔴 @rolldown/binding-linux-x64-gnu [#################.......] 19.35 MB 26.8%
🔴 ai                              [######..................] 6.65 MB 9.2%
🔴 zod                             [####....................] 5.07 MB 7.0%
🟠 undici                          [###.....................] 3.50 MB 4.9%
🟠 nitro                           [##......................] 2.41 MB 3.3%
🔴 Other installed packages        [#######.................] 7.77 MB 10.8%
Runtime dependencies (2)
Package Range Notes
nitro 3.0.260610-beta
undici 8.9.0
Peer dependencies (5)
Package Range Notes
@opentelemetry/api ^1.0.0 optional peer
ai catalog:
braintrust ^3.0.0 optional peer
just-bash ^3.0.0 optional peer
microsandbox ^0.5.0 optional peer
eve init install drill-down

eve init install details

  • Command: eve init my-agent
  • Package manager: npm
  • Installed footprint: 110.59 MB across 8656 installed files
  • Installed packages: 121 total (115 transitive-only)
  • dependencies: 4 direct packages totaling 43.26 MB
  • devDependencies: 2 direct packages totaling 5.04 MB
  • Other transitive package files: 62.29 MB

Installed footprint is measured from an isolated temporary eve init my-agent using the current packed eve tarball.

Heavy installed dependencies

  • @typescript/typescript-linux-x64: 27.95 MB (25.3%)
  • eve: 27.44 MB (24.8%)
  • @rolldown/binding-linux-x64-gnu: 19.35 MB (17.5%)
  • zod: 9.02 MB (8.2%)
  • ai: 6.65 MB (6.0%)
Installed footprint breakdown
Installed package size
🔴 @typescript/typescript-linux-x64 [########################] 27.95 MB 25.3%
🔴 eve                              [########################] 27.44 MB 24.8%
🔴 @rolldown/binding-linux-x64-gnu  [#################.......] 19.35 MB 17.5%
🔴 zod                              [########................] 9.02 MB 8.2%
🔴 ai                               [######..................] 6.65 MB 6.0%
🟠 undici                           [###.....................] 3.50 MB 3.2%
🔴 Other installed packages         [##############..........] 16.68 MB 15.1%
dependencies (4)
Package Range Installed size Share
@vercel/connect 0.4.3 141.0 kB 0.1%
ai ^7.0.38 6.65 MB 6.0%
eve file:eve-0.30.7.tgz 27.44 MB 24.8%
zod 4.4.3 9.02 MB 8.2%
devDependencies (2)
Package Range Installed size Share
@types/node 24.x 2.54 MB 2.3%
typescript 7.0.2 2.50 MB 2.3%
Function Drill-Down

Payload Size Graph

Unique function payload size and share of total
🔴 functions/.well-known/workflow/v1/flow.func     [########################] 8.62 MB 50.0%
🔴 functions/__server.func                         [########################] 8.62 MB 50.0%

Top Function Payloads

🟠 functions/.well-known/workflow/v1/flow.func • 1 public route • 8.62 MB
Metric Value
Public routes /.well-known/workflow/v1/flow
Runtime nodejs24.x
Handler index.mjs
Payload 8.62 MB
Function files 8.62 MB across 44 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.43 MB (28.2%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.43 MB 28.2%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.63 MB 18.9%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.4%
🟡 _chunks/sandbox.mjs             [########................] 769.0 kB 8.9%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.0%
🟠 Other bundled files             [#######################.] 2.38 MB 27.6%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x",
  "maxDuration": "max",
  "experimentalTriggers": [
    {
      "type": "queue/v2beta",
      "topic": "__eve776561746865722d6167656e74_wkf_workflow_*",
      "consumer": "default",
      "retryAfterSeconds": 5,
      "initialDelaySeconds": 0
    }
  ],
  "environment": {
    "WORKFLOW_PRECONDITION_GUARD": "1"
  }
}

🟠 functions/__server.func • 12 public routes, 1 internal alias • 8.62 MB
Metric Value
Public routes /
/eve/v1/callback/[token]
/eve/v1/connections/[name]/callback/[token]
/eve/v1/health
/eve/v1/info
/eve/v1/session
/eve/v1/session/[sessionId]
/eve/v1/session/[sessionId]/cancel
/eve/v1/session/[sessionId]/stream
/eve/v1/session/clear
/eve/v1/session/compact
/eve/v1/session/reset
Internal aliases /__server
Runtime nodejs24.x
Handler index.mjs
Payload 8.62 MB
Function files 8.62 MB across 44 files
Traced dependencies 0 B
Signal 🟠 Bundled file index.mjs is 2.43 MB (28.2%)

🟠 🔎 Dependency Analysis

📦 Bundled files:

Bundled file size
🟠 index.mjs                       [########################] 2.43 MB 28.2%
🟠 _chunks/runtime-artifacts.mjs   [################........] 1.63 MB 18.9%
🟡 _libs/undici.mjs                [##########..............] 980.5 kB 11.4%
🟡 _chunks/sandbox.mjs             [########................] 769.0 kB 8.9%
🟡 _libs/@ai-sdk/gateway+[...].mjs [####....................] 432.8 kB 5.0%
🟠 Other bundled files             [#######################.] 2.38 MB 27.6%

🧾 Vercel Config

{
  "handler": "index.mjs",
  "launcherType": "Nodejs",
  "shouldAddHelpers": false,
  "supportsResponseStreaming": true,
  "runtime": "nodejs24.x"
}

Build Timing: e2e/fixtures/agent-tools-sandbox

This is an informational timing measurement inside eve build, from preflight through publication. Output-size measurement and profile writing are excluded.

Build mode: deployable Vercel build with sandbox template prewarm included.

  • Build pipeline: 1.88 s -> 1.76 s (-112.1 ms) vs tasks/09-background-section-finalization (11528be).
  • Timing is informational: shared GitHub runners are too variable for a hard timing budget.
Detailed phase timings vs `tasks/09-background-section-finalization (11528be)`
Phase Baseline Current Delta
extension.check 8.9 ms 7.2 ms -1.7 ms
project.resolve 3.3 ms 2.3 ms -1.0 ms
workspace.create 0.8 ms 0.9 ms +0.1 ms
host.prepare 237.5 ms 184.6 ms -52.9 ms
vercel.service-prefix.resolve 1.4 ms 1.4 ms 0.0 ms
nitro.create 180.4 ms 176.9 ms -3.5 ms
sandbox.prewarm 274.9 ms 314.2 ms +39.3 ms
nitro.cache.prepare 0.2 ms 0.2 ms 0.0 ms
nitro.prepare 0.6 ms 0.6 ms 0.0 ms
nitro.public-assets 0.6 ms 0.6 ms 0.0 ms
nitro.prerender 0.4 ms 0.4 ms 0.0 ms
nitro.bundle 1.10 s 1.03 s -75.4 ms
nitro.cache.write 0.2 ms 0.3 ms +0.1 ms
vercel.workflow-function.materialize 56.9 ms 38.9 ms -18.0 ms
agent-summary.emit 0.5 ms 0.5 ms 0.0 ms
nitro.close 0.1 ms 0.1 ms 0.0 ms
output.publish 7.4 ms 8.4 ms +1.0 ms
workspace.remove 1.5 ms 1.6 ms +0.1 ms

Signed-off-by: Rui Conti <ruiconti@gmail.com>
@ruiconti

ruiconti commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by #1759: same seven evals plus a fix for the world-agnostic exclusivity-race flake seen in this PR's CI. The deterministic e2e-vercel fanout failure is a product bug, tracked in #1759's description.

@ruiconti ruiconti closed this Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant