Skip to content

fix: address lambda E2E test flaky timeout failures#213

Merged
ack-prow[bot] merged 1 commit intoaws-controllers-k8s:mainfrom
sapphirew:flaky-timeout-test
Apr 9, 2026
Merged

fix: address lambda E2E test flaky timeout failures#213
ack-prow[bot] merged 1 commit intoaws-controllers-k8s:mainfrom
sapphirew:flaky-timeout-test

Conversation

@sapphirew
Copy link
Copy Markdown
Contributor

@sapphirew sapphirew commented Apr 9, 2026

Issue #, if available:

Description of changes:

Under 33-way pytest-xdist parallel runs, controller reconciliation and resource deletion can exceed the acktest default timeouts (30s and 5s), causing non-deterministic test failures. Increase timeouts at call sites in all 7 lambda E2E test files without modifying acktest library defaults:

  1. Increase wait_resource_consumed_by_controller timeout at call sites: Change all calls from k8s.wait_resource_consumed_by_controller(ref) (using defaults: 3 periods × 10s = 30s) to k8s.wait_resource_consumed_by_controller(ref, wait_periods=10, period_length=10) (10 periods × 10s = 100s). This provides ~3× headroom over the current 30s default for parallel execution.

  2. Increase delete_custom_resource timeout at call sites: Change all calls from k8s.delete_custom_resource(ref) (using defaults: 1 period × 5s = 5s) to k8s.delete_custom_resource(ref, wait_periods=3, period_length=10) (3 periods × 10s = 30s). This provides 6× headroom over the current 5s default.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ack-prow ack-prow Bot requested review from jlbutler and knottnt April 9, 2026 01:28
@sapphirew sapphirew closed this Apr 9, 2026
@sapphirew sapphirew force-pushed the flaky-timeout-test branch from 4523795 to cf01c04 Compare April 9, 2026 06:00
@sapphirew sapphirew reopened this Apr 9, 2026
@sapphirew
Copy link
Copy Markdown
Contributor Author

/retest

@sapphirew sapphirew force-pushed the flaky-timeout-test branch from 32abec9 to 7beac7c Compare April 9, 2026 06:39
Copy link
Copy Markdown
Member

@michaelhtm michaelhtm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sapphirew
/lgtm

@ack-prow ack-prow Bot added the lgtm Indicates that a PR is ready to be merged. label Apr 9, 2026
@ack-prow
Copy link
Copy Markdown

ack-prow Bot commented Apr 9, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: michaelhtm, sapphirew

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ack-prow ack-prow Bot added the approved label Apr 9, 2026
@ack-prow ack-prow Bot merged commit 6f2da25 into aws-controllers-k8s:main Apr 9, 2026
8 checks passed
@sapphirew sapphirew deleted the flaky-timeout-test branch April 9, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants