-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ephemeral runner gets stuck in Successful state #3527
Comments
Hello! Thank you for filing an issue. The maintainers will triage your issue shortly. In the meantime, please take a look at the troubleshooting guide for bug reports. If this is a feature request, please review our contribution guidelines. |
Hey @katarzynainit, Can you please show the controller values.yaml file, so I can try to reproduce this issue. |
Hi, we are using forked arc-controller - code changes relate to skipping controller and listeners SA and RBAC creation based on three flags. https://gist.github.com/katarzynainit/d9e6ed4d3c6b95e929d73e2b1e8f7cc1 (flags for internal changes are marked in the values) We started to observe this issue on faster cluster, we didn't see them before (the same configuration, but different and slower cluster). It also happens from time to time only, so might be difficult to observe. |
Checks
Controller Version
0.9.0
Deployment Method
Helm
Checks
To Reproduce
Describe the bug
In controller logs I see that it already "Found the runner with the same name" - it looks like the controller is performing reconcile twice for the same ephemeralrunner in almost the same time, the second run "removes" runner and makes it hung.
The runner is eventually not created, and the ephemeral runner gets to stage Succeeded and stuck until workflow is cancelled.
We started to observe this behavior when we moved to faster cluster.
Describe the expected behavior
The controller should create runner always on ephemeral runner creation.
Additional Context
N/A
Controller Logs
Runner Pod Logs
The text was updated successfully, but these errors were encountered: