Skip to content

vault e2e: fix flaky batch response ordering assumption#21801

Merged
prashantkumar1982 merged 1 commit intodevelopfrom
fix/vault-test-response-ordering
Apr 1, 2026
Merged

vault e2e: fix flaky batch response ordering assumption#21801
prashantkumar1982 merged 1 commit intodevelopfrom
fix/vault-test-response-ordering

Conversation

@prashantkumar1982
Copy link
Copy Markdown
Contributor

@prashantkumar1982 prashantkumar1982 commented Mar 31, 2026

Summary

Fixes an intermittent test failure in the vault e2e basic_crud test where batch responses (create, update, delete) were asserted by positional index, but the Vault DON's OCR consensus does not guarantee response ordering.

Error observed:

expected: "main"
actual  : "alt"

Root cause: Responses[0] was assumed to correspond to namespaces[0] ("main"), but the DON returned "alt" first.

Fix: Replace positional Responses[i] checks with map-based lookups keyed by namespace. Applied to all three batch assertion blocks (create, update, delete). The "invalid" key entries in update/delete are matched by key name instead of by position.

The Vault DON processes batch requests through OCR consensus, which does
not guarantee the response array order matches the request array order.
The create, update, and delete test assertions were using positional
indexing (Responses[i] == namespaces[i]), causing intermittent failures
when the DON returned responses in a different order.

Replace positional checks with map-based lookups keyed by namespace,
so the assertions are order-independent.

Made-with: Cursor
@github-actions
Copy link
Copy Markdown
Contributor

👋 prashantkumar1982, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

@github-actions
Copy link
Copy Markdown
Contributor

✅ No conflicts with other open PRs targeting develop

@cl-sonarqube-production
Copy link
Copy Markdown

@trunk-io
Copy link
Copy Markdown

trunk-io bot commented Mar 31, 2026

Static BadgeStatic BadgeStatic BadgeStatic Badge

Failed Test Failure Summary Logs
Test_CRE_V2_EVM_GetTransactionByHash_Invalid_Hash_Regression The test failed without providing specific error details or reasons. Logs ↗︎
Test_CRE_V2_EVM_GetTransactionByHash_Invalid_Hash_Regression/[v2]_EVM.GetTransactionByHash_-_invalid_hash_fails_with_non-existent_hash The test failed because it did not find the expected user log message indicating a non-existent transaction hash. Logs ↗︎
Test_CRE_V2_EVM_GetTransactionByHash_Invalid_Hash_Regression/[v2]_EVM.GetTransactionByHash_-_invalid_hash_fails_with_short_hash The test failed because it did not find the expected user log message indicating an invalid hash error. Logs ↗︎

View Full Report ↗︎Docs

@Tofel Tofel self-requested a review April 1, 2026 07:10
@prashantkumar1982 prashantkumar1982 added this pull request to the merge queue Apr 1, 2026
@Tofel Tofel removed this pull request from the merge queue due to a manual request Apr 1, 2026
@prashantkumar1982 prashantkumar1982 added this pull request to the merge queue Apr 1, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Apr 1, 2026
@prashantkumar1982 prashantkumar1982 added this pull request to the merge queue Apr 1, 2026
Merged via the queue into develop with commit f8531a5 Apr 1, 2026
215 of 218 checks passed
@prashantkumar1982 prashantkumar1982 deleted the fix/vault-test-response-ordering branch April 1, 2026 17:10
prashantkumar1982 added a commit that referenced this pull request Apr 2, 2026
The Vault DON processes batch requests through OCR consensus, which does
not guarantee the response array order matches the request array order.
The create, update, and delete test assertions were using positional
indexing (Responses[i] == namespaces[i]), causing intermittent failures
when the DON returned responses in a different order.

Replace positional checks with map-based lookups keyed by namespace,
so the assertions are order-independent.

Made-with: Cursor
bolekk pushed a commit that referenced this pull request Apr 6, 2026
The Vault DON processes batch requests through OCR consensus, which does
not guarantee the response array order matches the request array order.
The create, update, and delete test assertions were using positional
indexing (Responses[i] == namespaces[i]), causing intermittent failures
when the DON returned responses in a different order.

Replace positional checks with map-based lookups keyed by namespace,
so the assertions are order-independent.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants