fix: add build failed condition when failure state reached#337
Merged
shreddedbacon merged 1 commit intomainfrom Dec 15, 2025
Merged
fix: add build failed condition when failure state reached#337shreddedbacon merged 1 commit intomainfrom
shreddedbacon merged 1 commit intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a critical bug where LagoonBuild CRs were not properly updating their status when build pods entered failure states like ImagePullBackOff or CrashLoopBackOff. Previously, the controller would clean up the failed pod and only update the build label, but not the status conditions and phase, leaving the build stuck in a pending/running state and blocking subsequent builds in the namespace.
Key Changes:
- Replaced
r.Update()withr.Patch()for updating the LagoonBuild resource to ensure atomic status updates - Added status condition setting using
BuildStepToStatusConditionshelper to properly track build step failures - Added phase field to the status update to correctly reflect the failed state
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
7bc2c7d to
7744412
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Checklist
If a build pod reaches an
ImagePullBackOfforCrashLoopBackOffcondition, the build does not update its status in the API to reflect, and theLagoonBuildCR remains in a pending/running state blocking subsequent builds in that namespace from starting.Right now, the controller captures the failure, but the
LagoonBuildCR does not get updated properly for the controller to detect it. So the build pod is cleaned up, and the controller thinks everything is ok, when it isn't.To observe this, spin up a local-stack and deploy one of the demo environments, watch it complete successfully. Now log into the registry in the local-stack and delete the
library/build-deploy-image, this will force theImagePullBackOffcondition on the next build. Now trigger another build and you'll see the build pod start, hit the condition. The controller will clean it up and mark the build as "failed". However, theLagoonBuildCR for that build will show that it isPendingwith arunningstate. If you trigger another deployment, it will not progress due to the previous build thinking it is still running.Now, if you delete the build CR that is stuck, the next build will start, but will then do the same thing.
To verify the fix, delete all the
LagoonBuildCRs from the namespace. Replace the controller image with the one in this PR and repeat the steps. You'll see the builds now no longer get stuck in pending, and report back as failed correctly.