Skip to content

fix(e2e): remove mocks for real APIs#38

Merged
vitramir merged 21 commits intomainfrom
noa/issue-36-2
Mar 24, 2026
Merged

fix(e2e): remove mocks for real APIs#38
vitramir merged 21 commits intomainfrom
noa/issue-36-2

Conversation

@casey-brooks
Copy link
Copy Markdown
Contributor

Summary

  • map ChatGateway/AgentsGateway wire fields in adapters and mock plugin
  • remove Playwright API mocks and rework e2e flows to create real chats
  • update devspace sync checks for the new e2e layout

Testing

Refs #36

@casey-brooks casey-brooks requested a review from a team as a code owner March 23, 2026 12:17
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • map ChatGateway/AgentsGateway wire fields in adapters and mock plugin
  • remove Playwright API mocks and rework e2e flows to create real chats
  • update devspace sync checks for the new e2e layout

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • pnpm test (2 passed)
  • pnpm build
  • E2E_BASE_URL=http://localhost:3000 pnpm test:e2e (5 passed)

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • switch e2e ConnectRPC helpers to use the browser context request API
  • update e2e tests to pass page.context() so auth cookies apply

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • pnpm test (2 passed)
  • pnpm build
  • E2E_BASE_URL=http://localhost:3000 pnpm test:e2e (5 passed)

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work removing the Playwright mocks and wiring up real API adapters. The proto wire field mapping is well-structured and the e2e test setup via BrowserContext.request is a clean approach.

Requesting changes for two major issues:

  1. Silent error suppression in expectConversationsListVisibletry/catch around a Playwright assertion is fragile and masks failures. Use locator.or() instead.

  2. Near-duplicate test logicconversations-list.spec.ts and conversation-create.spec.ts have nearly identical "create conversation via UI" flows. Deduplicate or remove the redundant test.

  3. Test illusion on empty agents — Tests silently pass when no agents are available, hiding the fact that the core flow was never exercised. Use test.skip() or fail explicitly.

Additional minor/nit feedback on naming (mapConversationResponse), duplicated wire types (AgentWire in 3 files), removed test coverage (redirect/navigation), and silent filtering in listAgents.

Comment thread test/e2e/conversations-list.spec.ts Outdated
Comment thread test/e2e/conversations-list.spec.ts Outdated
Comment thread test/e2e/chat-api.ts Outdated
Comment thread src/api/modules/conversations.ts Outdated
Comment thread devspace.yaml Outdated
Comment thread test/e2e/conversations-list.spec.ts Outdated
Comment thread src/api/modules/agents.ts Outdated
Comment thread test/e2e/conversation-create.spec.ts Outdated
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • centralize AgentWire definition and tighten listAgents validation
  • rename chat payload validator and update e2e list routing coverage
  • simplify conversations list visibility check and add skip for empty-agent create flow
  • include chat-api in devspace sync readiness

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • pnpm test (2 passed)
  • pnpm build
  • E2E_BASE_URL=http://localhost:3000 pnpm test:e2e (6 passed)

noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 7 prior review items are resolved and verified in code:

  1. try/catchlocator.or() in expectConversationsListVisible
  2. ✅ Duplicate "create via UI" test removed from conversations-list.spec.ts
  3. listAgents now throws on invalid payloads instead of silently filtering
  4. mapConversationResponsevalidateChatPayload
  5. chat-api.ts added to devspace sync readiness check
  6. redirects root to /conversations and navigates to conversation detail tests restored
  7. test.skip() for empty agents in conversation-create.spec.ts
  8. AgentWire extracted to src/api/types/wire/agents.ts and shared across all three consumers

One nit left on dead code after test.skip() — non-blocking.

LGTM.

Comment thread test/e2e/chat-create.spec.ts Outdated
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • updated conversation hooks/pages to use the chat API shapes and removed status toggling
  • aligned mock API data with chat + agent proto shapes and reminder chat IDs
  • refreshed e2e helper types for the new agent response shape

Testing

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • added multi-user Playwright fixture plus chat exchange e2e spec
  • exposed /chats routes and updated message test ids for chat-based selectors
  • expanded devspace file sync readiness checks for new e2e files

Testing

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Renamed conversation hooks/components/pages/utils/mock data to chat counterparts, including routes/test IDs and updated E2E specs.
  • Updated mock API and routing to align with /chats while retaining /conversations redirects.
  • Resolved ChatMessage type name collision in Chats page imports.

Tests:

  • pnpm lint (no errors)
  • pnpm typecheck (pass)
  • pnpm test (passed: 2, failed: 0, skipped: 0)
  • pnpm build (success; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (passed: 7, failed: 0, skipped: 0)

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The full codebase rename from conversation*chat* is clean and thorough. Types now match proto exactly, all adapters are removed, and the API modules are clean pass-throughs. The socket boundary correctly preserves server-defined wire event names while translating to internal chat* naming. Mock plugin, mock data, components, hooks, pages, routes, storage keys, and test IDs are all consistently renamed. The UpdateChatStatus RPC was properly removed. Backward-compat route redirects are in place.

Requesting changes for one issue:

  1. [major] chat-exchange.spec.ts — User B is not a participant in the chat. createChat uses a random UUID, not User B's identity. The test doesn't exercise a real two-participant exchange. If the server enforces access control, the test will fail.

Additionally:

  • [minor] multi-user-fixtures.ts — duplicate fixture setup should be extracted into a helper.
  • [nit] chat-create.spec.ts — dead if block after test.skip() (carried over from prior review).

Comment thread test/e2e/chat-exchange.spec.ts Outdated
Comment thread test/e2e/multi-user-fixtures.ts
Comment thread test/e2e/chat-create.spec.ts Outdated
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Resolved user participant IDs for chat-exchange by decoding OIDC session data with fallback to user email, and removed random participant generation.
  • Refactored multi-user fixtures into a shared context helper.
  • Removed dead skip logic from chat-create spec.

Tests:

  • pnpm lint (no errors)
  • pnpm typecheck (pass)
  • pnpm test (passed: 2, failed: 0, skipped: 0)
  • pnpm build (success; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (passed: 7, failed: 0, skipped: 0)

noa-lucent
noa-lucent previously approved these changes Mar 23, 2026
Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three items verified in code:

  1. ✅ User B participantresolveUserId reads the OIDC sub claim from session storage with JWT decode and email fallbacks, then passes it to createChat. User B is now a proper participant. The resolution logic is well-layered and correctly defensive as boundary code.
  2. ✅ Duplicate fixture logiccreateUserContext(browser, email) extracted; both fixtures are now clean three-liners.
  3. ✅ Dead codeif block after test.skip() removed.

LGTM — clean rename, proto-aligned types, proper boundary/internal separation, and solid e2e coverage.

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Switched e2e Connect RPC helpers to use Page context and attach Authorization bearer tokens from sessionStorage when available.
  • Updated e2e specs to pass Page into chat-api helpers.
  • Kept e2e helpers compatible with non-OIDC local runs by skipping auth headers when no OIDC session entry exists.

Tests:

  • pnpm lint (no errors)
  • pnpm typecheck (pass)
  • pnpm test (passed: 2, failed: 0, skipped: 0)
  • pnpm build (success; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (passed: 7, failed: 0, skipped: 0)

rowan-stein added a commit to agynio/bootstrap that referenced this pull request Mar 24, 2026
Picks up fix(oidc): remove access token aud check (agynio/gateway#96).
This unblocks real-API e2e tests in agynio/chat-app#38.
vitramir pushed a commit to agynio/bootstrap that referenced this pull request Mar 24, 2026
Picks up fix(oidc): remove access token aud check (agynio/gateway#96).
This unblocks real-API e2e tests in agynio/chat-app#38.
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Adjusted listAgents to treat missing proto3 repeated fields as empty arrays.
  • Added sessionStorage key/token diagnostics and response-body logging for Connect RPC calls.

Tests:

  • pnpm lint (no errors)
  • pnpm typecheck (pass)
  • pnpm test (passed: 2, failed: 0, skipped: 0)
  • pnpm build (success; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (passed: 7, failed: 0, skipped: 0)

rowan-stein added a commit to agynio/bootstrap that referenced this pull request Mar 24, 2026
Chat v0.1.0 required x-tenant-id gRPC metadata which the gateway never
set and was a dead field (extracted but unused). Chat v0.2.0 (PR #17)
removes this requirement, keeping only x-identity-id and x-identity-type.

Unblocks e2e tests in agynio/chat-app#38.
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Added organizationId config support and pass it through the agents list hook.
  • Removed e2e agent list mocking, added E2E org constant, and seed agents with org id.
  • Ensured playwright config sets VITE_ORGANIZATION_ID for e2e runs.

Tests:

  • pnpm lint (pass)
  • pnpm typecheck (pass)
  • pnpm test (pass: 1 file, 2 tests)
  • pnpm build (pass; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (pass: 8 tests; dev server started with VITE_API_BASE_URL=/api VITE_ORGANIZATION_ID=00000000-0000-4000-a000-000000000001)

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Reverted org-id wiring changes and removed the chat-create e2e test plus unused agent helpers.
  • Confirmed no e2e mocks/intercepts remain; remaining tests hit real APIs only.

Tests:

  • pnpm lint (pass)
  • pnpm typecheck (pass)
  • pnpm test (pass: 1 file, 2 tests)
  • pnpm build (pass; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (pass: 6 tests; dev server via VITE_API_BASE_URL=/api pnpm dev --host 0.0.0.0 --port 5173 --strictPort)

@rowan-stein
Copy link
Copy Markdown
Collaborator

Ready for re-review

Changes since last review:

  • Reverted the broken VITE_ORGANIZATION_ID approach
  • Removed chat-create.spec.ts — cannot work without the org fix (separate work)
  • Zero mocks remain — all e2e tests hit real APIs
  • 6 e2e tests pass locally: sign-in, chats-list ×3, chat-detail, chat-exchange

The organizations gap (frontend has no org context → ListAgents can't work end-to-end) will be addressed in a follow-up PR with cross-repo changes.

Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The env-var revert, chat-create.spec.ts removal (blocked by org context gap), and signature change from BrowserContext to Page (to enable getAccessToken) are reasonable.

The CreateAgent/DeleteAgent additions in the vite mock plugin are clean.

Requesting changes for three issues:

  1. [major] devspace.yaml still references deleted chat-create.spec.ts — the file sync readiness check will always time out in CI.
  2. [major] Duplicate session storage iterationreadOidcSession and getAccessToken both iterate and parse the same oidc.user: session storage blob independently. Consolidate into a single helper.
  3. [major] Debug console.log statements — three log lines leak access tokens, session storage keys, and response bodies into test output. Remove them.

Comment thread devspace.yaml Outdated
Comment thread test/e2e/chat-api.ts Outdated
Comment thread test/e2e/chat-api.ts
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary:

  • Removed deleted chat-create spec from devspace e2e sync checks.
  • Consolidated OIDC session parsing into a single helper and removed debug token/key logging in e2e API helpers.

Tests:

  • pnpm lint (pass)
  • pnpm typecheck (pass)
  • pnpm test (pass: 1 file, 2 tests)
  • pnpm build (pass; chunk size warning)
  • E2E_BASE_URL=http://localhost:5173 pnpm test:e2e (pass: 6 tests; dev server via VITE_API_BASE_URL=/api pnpm dev --host 0.0.0.0 --port 5173 --strictPort)

noa-lucent
noa-lucent previously approved these changes Mar 24, 2026
Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three items verified in code:

  1. devspace.yaml — both chat-create.spec.ts references removed from readiness check
  2. readOidcSession consolidated — single helper now returns { profileSub, profileEmail, idToken, accessToken }; duplicate getAccessToken deleted; postConnect consumes session.accessToken directly
  3. ✅ All debug console.log statements removed; error body included in thrown Error message instead

LGTM.

emerson-gray added a commit to agynio/bootstrap that referenced this pull request Mar 24, 2026
Chat chart v0.2.1 (agynio/chat#18) defaults threadsAddress to
threads:50051, following the gateway chart convention where
inter-service gRPC addresses use in-cluster DNS defaults.

Remove the v0.1.0-specific service.port and threads.address overrides
— the chart defaults handle both correctly.

This also removes the x-tenant-id requirement (dropped in chat v0.2.0,
agynio/chat#17), unblocking real-API e2e tests in agynio/chat-app#38.
vitramir pushed a commit to agynio/bootstrap that referenced this pull request Mar 24, 2026
* chore: bump chat chart version to 0.2.0

Chat v0.1.0 required x-tenant-id gRPC metadata which the gateway never
set and was a dead field (extracted but unused). Chat v0.2.0 (PR #17)
removes this requirement, keeping only x-identity-id and x-identity-type.

Unblocks e2e tests in agynio/chat-app#38.

* chore: bump chat chart version to 0.2.1

Chat chart v0.2.1 (agynio/chat#18) defaults threadsAddress to
threads:50051, following the gateway chart convention where
inter-service gRPC addresses use in-cluster DNS defaults.

Remove the v0.1.0-specific service.port and threads.address overrides
— the chart defaults handle both correctly.

This also removes the x-tenant-id requirement (dropped in chat v0.2.0,
agynio/chat#17), unblocking real-API e2e tests in agynio/chat-app#38.

* ci: re-trigger after chat v0.2.1 chart published

* fix: drop v prefix from chat image tag resolution

The chat Docker workflow uses docker/metadata-action with
semver pattern={{version}} which strips the v prefix from tags.
Tag v0.2.1 produces image ghcr.io/agynio/chat:0.2.1 (no v).

Align with gateway convention: use chart_version directly
instead of format("v%s", chart_version).

* chore: bump chat chart to 0.2.2 (numeric UID fix)

* fix: revert accidental notifications_chart_version bump

---------

Co-authored-by: Emerson Gray <emerson.gray@agyn.io>
@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • Resolve a default participant ID in createChat to allow self-chat creation when none is provided.

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • PATH=/workspace/bin:$PATH pnpm test:unit (Tests: 2 passed, 0 failed, 0 skipped)
  • pnpm build

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • Added /api/me identity resolution helper for e2e chat setup and switched chat creation to use identity UUIDs.
  • Updated chat exchange test imports/fixtures to rely on identity IDs and dropped the unused USER_B_EMAIL export.

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • PATH=/workspace/bin:$PATH pnpm test:unit (Tests: 2 passed, 0 failed, 0 skipped)
  • pnpm build

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • Added browser API response logging in e2e fixtures to surface failing API calls.
  • Added waits for GetMessages/GetChats responses and increased chat content visibility timeouts to reduce flakiness.

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • PATH=/workspace/bin:$PATH pnpm test:unit (Tests: 2 passed, 0 failed, 0 skipped)
  • pnpm build

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • Normalized chat API responses so repeated fields default to empty arrays and added message/fileId safeguards in chat hooks.
  • Removed temporary browser API response logging from e2e fixtures.

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • PATH=/workspace/bin:$PATH pnpm test:unit (Tests: 2 passed, 0 failed, 0 skipped)
  • pnpm build

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • Removed the redundant GetChats response wait from the chat detail navigation test.

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • PATH=/workspace/bin:$PATH pnpm test:unit (Tests: 2 passed, 0 failed, 0 skipped)
  • pnpm build

@rowan-stein
Copy link
Copy Markdown
Collaborator

CI Green — 6/6 e2e passing ✅

Changes since last review approval:

  1. createChat self-participant fallback — when no participant specified, resolves caller's own identity via resolveIdentityId (was sending empty participantIds)
  2. Identity resolution via /api/me — replaced OIDC sub (Zitadel 20-char ID) with gateway-resolved identity UUID. Removed old resolveUserId/decodeJwtSubject helpers.
  3. Proto3 zero-value normalizationfileIds and participants arrays defaulted to [] in API module (proto3 JSON omits empty repeated fields)
  4. Removed waitForResponse racechats-list navigate test was waiting for GetChats response that never fires (client-side navigation, data cached)
  5. Increased e2e assertion timeouts to 15s for chat content visibility (cold k3d cluster)

Requesting re-review.

noa-lucent
noa-lucent previously approved these changes Mar 24, 2026
Copy link
Copy Markdown

@noa-lucent noa-lucent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes since last approval look good:

  • resolveIdentityId via /api/me — cleaner and authoritative compared to OIDC sub parsing. Proper boundary validation on the response.
  • Proto3 zero-value normalizationnormalizeChat/normalizeMessage are correctly placed at the API module boundary, ensuring internal code always sees non-optional arrays. Clean pattern.
  • createChat self-participant fallback — reasonable default behavior for single-user test flows.
  • Timeout increases — pragmatic for cold k3d clusters.
  • waitForResponse removal from chats-list — correct, since client-side navigation uses cached data.

Two minor items noted (non-blocking):

  1. readOidcSession still extracts profileSub, profileEmail, idToken — all dead since resolveUserId was deleted. Trim to just accessToken.
  2. waitForResponse after await goto() is a race — start listening before navigation or remove if the toBeVisible assertion is the real gate.

LGTM.

Comment thread test/e2e/chat-api.ts Outdated
Comment on lines +27 to +30
profileSub: string | null;
profileEmail: string | null;
idToken: string | null;
accessToken: string | null;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] profileSub, profileEmail, and idToken are no longer consumed by any caller — the only consumer was the deleted resolveUserId. Now both postConnect and resolveIdentityId only read accessToken.

Trim the type and page.evaluate to only extract what's needed:

type OidcStorageSnapshot = {
  accessToken: string | null;
};

This removes ~10 lines of dead parsing logic. If these fields are needed later, they can be re-added.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed — trimmed readOidcSession to only extract accessToken. Dead fields removed.

Comment thread test/e2e/chat-detail.spec.ts Outdated
Comment on lines +11 to +14
await page.waitForResponse(
(resp) => resp.url().includes('GetMessages') && resp.status() === 200,
{ timeout: 15000 },
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[minor] waitForResponse is called after await page.goto() returns (which waits for the load event by default). If the GetMessages response completes during navigation — before waitForResponse starts listening — it will be missed, and the call will either time out or race.

The safe pattern is to start listening before triggering navigation:

const messagesLoaded = page.waitForResponse(
  (resp) => resp.url().includes('GetMessages') && resp.status() === 200,
  { timeout: 15000 },
);
await page.goto(`/chats/${chatId}`);
await messagesLoaded;

Same applies in chat-exchange.spec.ts (4 occurrences). That said, the toBeVisible assertion with 15s timeout is the actual correctness gate here, so this is non-blocking — but the waitForResponse calls as written are misleading since they may silently no-op.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed — waitForResponse now starts before page.goto() in both chat-detail.spec.ts and chat-exchange.spec.ts.

@casey-brooks
Copy link
Copy Markdown
Contributor Author

Summary

  • Simplified OIDC session parsing to only capture access tokens used in e2e API helpers.
  • Updated chat detail navigation tests to wait for GetMessages before navigation to avoid race conditions.

Testing

  • pnpm lint (no errors)
  • pnpm typecheck
  • PATH=/workspace/bin:$PATH pnpm test:unit (Tests: 2 passed, 0 failed, 0 skipped)
  • pnpm build

@vitramir vitramir merged commit 41213e8 into main Mar 24, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants