Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

check timestamps before waiting for agent rollout #5698

Merged
merged 2 commits into from
Apr 29, 2024

Conversation

ibrokethecloud
Copy link
Contributor

IMPORTANT: Please do not create a Pull Request without creating an issue first.

Problem:

PR attempts to address random upgrade fails where the upgrade fails due to the following error:

Wait for cluster settling down...
CAPI cluster fleet-local/local is provisioned (current generation: 838).
cluster.fleet.cattle.io/local patched
Error from server (NotFound): deployments.apps "fleet-agent" not found

This is caused by a delay in rollout of fleet-agent, where the rollout happens when the check for fleet-agent rollout status is running, and deployment gets recreated.

Solution:

The fix attempts to use original fleet-agent creation timestamp before fleet cluster object is patched, and ensures new fleet-agent deployment has a newer creation timestamp before rollout status is checked.

Related Issue:

Test plan:

@ibrokethecloud
Copy link
Contributor Author

should address: #5690

@bk201
Copy link
Member

bk201 commented Apr 29, 2024

@bk201 bk201 requested review from starbops and bk201 April 29, 2024 02:54
Copy link
Member

@starbops starbops left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The upgrade went well on a 2-node cluster. LGTM, thank you.

Copy link
Member

@bk201 bk201 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@bk201 bk201 merged commit cc1c2ad into harvester:master Apr 29, 2024
5 checks passed
@bk201
Copy link
Member

bk201 commented Apr 29, 2024

@Mergifyio backport v1.2 v1.3

Copy link

mergify bot commented Apr 29, 2024

backport v1.2 v1.3

🛑 Command backport v1.2 v1.3 cancelled because of a new backport command with different arguments

@bk201
Copy link
Member

bk201 commented Apr 29, 2024

@Mergifyio backport v1.2 v1.3

Copy link

mergify bot commented Apr 29, 2024

backport v1.2 v1.3

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Apr 29, 2024
(cherry picked from commit cc1c2ad)

# Conflicts:
#	package/upgrade/upgrade_manifests.sh
mergify bot pushed a commit that referenced this pull request Apr 29, 2024
ibrokethecloud added a commit that referenced this pull request Apr 30, 2024
(cherry picked from commit cc1c2ad)

fixed rebase conflicts
starbops pushed a commit that referenced this pull request Apr 30, 2024
starbops pushed a commit that referenced this pull request Apr 30, 2024
(cherry picked from commit cc1c2ad)

fixed rebase conflicts
bk201 pushed a commit to bk201/harvester that referenced this pull request May 2, 2024
(cherry picked from commit cc1c2ad)

fixed rebase conflicts
bk201 pushed a commit that referenced this pull request May 2, 2024
(cherry picked from commit cc1c2ad)

fixed rebase conflicts
bk201 pushed a commit to bk201/harvester that referenced this pull request May 23, 2024
bk201 pushed a commit that referenced this pull request May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants