Skip to content

Further E2E tests for resume#665

Merged
SteveSandersonMS merged 2 commits intomainfrom
stevesa/more-resume-e2e-tests
Mar 4, 2026
Merged

Further E2E tests for resume#665
SteveSandersonMS merged 2 commits intomainfrom
stevesa/more-resume-e2e-tests

Conversation

@SteveSandersonMS
Copy link
Contributor

Our E2E coverage for resume, and especially its interaction with streaming, was a bit inconsistent.

This expands coverage and makes it consistent across all languages.

@SteveSandersonMS SteveSandersonMS requested a review from a team as a code owner March 4, 2026 15:31
Copilot AI review requested due to automatic review settings March 4, 2026 15:31
@github-actions
Copy link
Contributor

github-actions bot commented Mar 4, 2026

✅ Cross-SDK Consistency Review: APPROVED

I've reviewed PR #665 for cross-language SDK consistency, and I'm happy to report this PR demonstrates excellent consistency across all four SDK implementations.

Summary of Changes

This PR expands E2E test coverage for session resume and streaming functionality consistently across all SDKs (Node.js, Python, Go, and .NET). The changes include:

1. Enhanced Session Resume Tests

  • All four SDKs now verify that resumed sessions maintain conversational state by adding a follow-up "double that" test in both resume scenarios (same client and new client)
  • This ensures that resume isn't just reconnecting, but truly preserving context

2. New Streaming Fidelity Test Suite

  • Created dedicated streaming fidelity test files for all four SDKs with identical coverage:
    • should_produce_delta_events_when_streaming_is_enabled - Verifies deltas are emitted and come before final message
    • should_not_produce_deltas_when_streaming_is_disabled - Verifies no deltas when streaming is off
    • should_produce_deltas_after_session_resume - Verifies streaming can be toggled on during resume (new client with streaming=true)

3. Test Organization Improvement

  • Moved streaming-related tests from session/ snapshots to dedicated streaming_fidelity/ snapshots for better organization

Cross-SDK Consistency Verification ✅

I verified that all four SDKs have equivalent test implementations:

Test Scenario Node.js Python Go .NET
Resume + stateful conversation
Streaming deltas when enabled
No deltas when disabled
Resume with streaming toggle

The test implementations are remarkably consistent:

  • Same test prompts across all languages ("Count from 1 to 5", "What is 3 + 6?", "double that")
  • Same assertions (delta events exist, have content, come before final message)
  • Same test flow (create → send → verify deltas → destroy)
  • Language-idiomatic naming respected (camelCase vs snake_case vs PascalCase)

No consistency issues found. This PR maintains excellent feature parity across all SDK implementations. 🎉

AI generated by SDK Consistency Review Agent

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands cross-SDK E2E coverage around session resume and its interaction with streaming delta events, aiming to make the scenarios consistent across Node, Python, Go, and .NET using the shared replay-proxy snapshot mechanism.

Changes:

  • Added/extended “continue conversation after resume” assertions across SDKs and updated the corresponding session snapshots.
  • Introduced dedicated Streaming Fidelity E2E test suites (Python/Go/.NET) and a new snapshot for “deltas after resume”.
  • Removed older session-level streaming tests/snapshots (with an apparent intent to consolidate streaming coverage under streaming_fidelity).

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
test/snapshots/streaming_fidelity/should_produce_deltas_after_session_resume.yaml Adds a shared snapshot conversation for streaming deltas after resuming a session.
test/snapshots/session/should_resume_a_session_using_the_same_client.yaml Extends snapshot to include a follow-up turn after resume.
test/snapshots/session/should_resume_a_session_using_a_new_client.yaml Extends snapshot to include a follow-up turn after resume.
test/snapshots/session/should_receive_streaming_delta_events_when_streaming_is_enabled.yaml Removes a session-category snapshot previously used by Node session E2E tests.
test/snapshots/session/should_pass_streaming_option_to_session_creation.yaml Removes a session-category snapshot previously used by Node session E2E tests.
python/e2e/test_streaming_fidelity.py Adds new Python streaming fidelity E2E tests, including “deltas after resume”.
python/e2e/test_session.py Adds assertions that resumed sessions remain stateful; removes older streaming-related tests.
nodejs/test/e2e/streaming_fidelity.test.ts Adds Node test for “deltas after resume” (new client resume scenario).
nodejs/test/e2e/session.test.ts Adds “continue conversation statefully” assertions for resume tests.
go/internal/e2e/streaming_fidelity_test.go Adds Go streaming fidelity E2E tests including “deltas after resume”.
go/internal/e2e/session_test.go Adds “continue conversation statefully” checks for resume tests.
dotnet/test/StreamingFidelityTests.cs Adds .NET streaming fidelity E2E tests including “deltas after resume”.
dotnet/test/SessionTests.cs Adds “continue conversation statefully” checks for resume tests; removes older streaming tests.

devm33
devm33 previously approved these changes Mar 4, 2026
- Remove unused 'answer' variable in Python streaming fidelity test
- Add onTestFinished cleanup for newClient in Node.js streaming resume test
- Remove streaming tests from Node.js session.test.ts (were still referencing deleted session/ snapshots)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@SteveSandersonMS SteveSandersonMS force-pushed the stevesa/more-resume-e2e-tests branch from b615abb to aaac89c Compare March 4, 2026 15:54
@SteveSandersonMS SteveSandersonMS merged commit 93ca61a into main Mar 4, 2026
35 checks passed
@SteveSandersonMS SteveSandersonMS deleted the stevesa/more-resume-e2e-tests branch March 4, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants