Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Realtime compositions E2E tests flaky #5228

Open
Tracked by #4828
, #5671
...
negz opened this issue Jan 12, 2024 · 5 comments · Fixed by #5276
Open
Tracked by #4828
, #5671
...

Realtime compositions E2E tests flaky #5228

negz opened this issue Jan 12, 2024 · 5 comments · Fixed by #5276
Assignees
@negz
Labels
bug Something isn't working composition e2e flaky-test realtime-composition
Milestone
v1.17

Comments

@negz
Copy link
Member

negz commented Jan 12, 2024

What happened?

Anecdotally the realtime compositions E2E test seems to fail a lot more often than it passes. This means I'm becoming used to seeing the little ❌ next to every Crossplane PR (and on master). Ideally it would not be so flaky.

How can we reproduce it?

Run the E2E in GitHub Actions a few times.

What environment did it happen in?

Crossplane version:
@negz negz mentioned this issue Jan 12, 2024
Open
@jbw976 jbw976 mentioned this issue Jan 22, 2024
Merged
6 tasks
negz added a commit to negz/crossplane that referenced this issue Jan 23, 2024
@negz
Bump crossplane-runtime
f4f3aa9 
Bring in crossplane/crossplane-runtime#641 to
see whether it helps with crossplane#5228.

Signed-off-by: Nic Cope <nicc@rk0n.org>
@negz negz mentioned this issue Jan 23, 2024
Merged
6 tasks
@negz negz reopened this Jan 23, 2024
@negz
Copy link
Member Author

negz commented Jan 23, 2024

FYI @sttts still seeing flakes after bumping runtime to include crossplane/crossplane-runtime#641.

@sttts
Copy link
Contributor

sttts commented Jan 24, 2024

@negz yes, was expecting that. Matches feedback in #5151. Will take another look when I have time.

negz added a commit to negz/crossplane that referenced this issue Jan 27, 2024
@negz
Don't run Realtime Composition E2E tests in CI
53b143f 
Per crossplane#5228 we know these
tests fail most of the time. Running them on every CI job isn't helping
catch anything we don't already know. It does make it harder to notice
when other things fail, since we're becoming used to seeing the red x.

I think we should disable these until they're less flaky. They're
testing an alpha feature that is off by default.

Signed-off-by: Nic Cope <nicc@rk0n.org>
@negz negz mentioned this issue Jan 27, 2024
Merged
6 tasks
negz added a commit to negz/crossplane that referenced this issue Jan 31, 2024
@negz
Bump crossplane-runtime
370854c 
Bring in crossplane/crossplane-runtime#641 to
see whether it helps with crossplane#5228.

Signed-off-by: Nic Cope <nicc@rk0n.org>
negz added a commit to negz/crossplane that referenced this issue Jan 31, 2024
@negz
Don't run Realtime Composition E2E tests in CI
27b9920 
Per crossplane#5228 we know these
tests fail most of the time. Running them on every CI job isn't helping
catch anything we don't already know. It does make it harder to notice
when other things fail, since we're becoming used to seeing the red x.

I think we should disable these until they're less flaky. They're
testing an alpha feature that is off by default.

Signed-off-by: Nic Cope <nicc@rk0n.org>
@github-actions GitHub Actions
Copy link

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 14 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

@github-actions github-actions bot added the stale label Apr 24, 2024
@jbw976
Copy link
Member

jbw976 commented Apr 24, 2024

/fresh

@github-actions github-actions bot removed the stale label Apr 24, 2024
@jbw976 jbw976 mentioned this issue May 10, 2024
Open
@negz negz self-assigned this May 14, 2024
@negz negz mentioned this issue May 14, 2024
Merged
6 tasks
@negz
Copy link
Member Author

negz commented May 21, 2024

#5651 might fix this - let's monitor for a while.

@jbw976 jbw976 added this to the v1.17 milestone May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@negz negz

Labels
bug Something isn't working composition e2e flaky-test realtime-composition
Projects
Crossplane Roadmap
Status: Backlog
Milestone
v1.17
Development

Successfully merging a pull request may close this issue.

Bump crossplane-runtime negz/crossplane
3 participants
@sttts @negz @jbw976