Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add retry logic to libicu download #3748

Merged
merged 5 commits into from
Feb 12, 2024

Conversation

jkruse14
Copy link
Contributor

@jkruse14 jkruse14 commented Feb 2, 2024

Background


There seems to be a race condition some where with the user-data script and the EC2 starting up and locking RPM. This issue is seen elsewhere, not necessarily with this repo's user-data, see the following:

Amazon Linux 2023 - issue with installing packages with cloud-init

dnf/yum both fails while being executed on instance bootstrap on Amazon Linux 2023

Also, #3741

Changes Made


Added a loop to retry if the rpm lock file is found which sleeps for 5 seconds then retries again with a total of 5 iterations.

Testing


Spun up an instance and ran the script directly (using sudo the second time because this was not run via user-data)
image

@jkruse14
Copy link
Contributor Author

jkruse14 commented Feb 8, 2024

@npalm - any chance we could get eyes on this?

Copy link
Member

@npalm npalm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks good. Need to test asap

@jkruse14
Copy link
Contributor Author

Thanks, looks good. Need to test asap

@npalm - Anything you need from me for testing?

Copy link
Member

@npalm npalm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, a small suggestion

modules/runners/templates/install-runner.sh Outdated Show resolved Hide resolved
Co-authored-by: Niek Palm <npalm@users.noreply.github.com>
@jkruse14 jkruse14 requested a review from npalm February 12, 2024 15:15
@npalm npalm merged commit 1b4597b into philips-labs:main Feb 12, 2024
3 checks passed
@jkruse14 jkruse14 deleted the feat/libicu-retry branch February 12, 2024 17:16
npalm pushed a commit that referenced this pull request Feb 19, 2024
🤖 I have created a release *beep* *boop*
---


##
[5.7.1](v5.7.0...v5.7.1)
(2024-02-13)


### Bug Fixes

* add retry logic to runner start script for libicu download
([#3748](#3748))
([1b4597b](1b4597b))
* **lambda:** bump the aws group in /lambdas with 6 updates
([#3762](#3762))
([e43fd80](e43fd80))
* **lambda:** bump the octokit group in /lambdas with 1 update
([#3763](#3763))
([77586c5](77586c5))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).

Co-authored-by: forest-releaser[bot] <80285352+forest-releaser[bot]@users.noreply.github.com>
npalm added a commit that referenced this pull request Mar 14, 2024
…r dnf operations (#3787)

## Background
---
This is a continuation of work done in #3748 

There seems to be a race condition some where with the user-data script
and the EC2 starting up and locking RPM. This issue is seen elsewhere,
not necessarily with this repo's user-data, see the following:

[Amazon Linux 2023 - issue with installing packages with
cloud-init](https://repost.aws/questions/QU_tj7NQl6ReKoG53zzEqYOw/amazon-linux-2023-issue-with-installing-packages-with-cloud-init)

[dnf/yum both fails while being executed on instance bootstrap on Amazon
Linux
2023](https://repost.aws/questions/QUgNz4VGCFSC2TYekM-6GiDQ/dnf-yum-both-fails-while-being-executed-on-instance-bootstrap-on-amazon-linux-2023)

Also,
#3741

## Changes Made
---
Added a loop to retry if the rpm lock file is found which sleeps for 5
seconds then retries again with a total of 5 iterations. This logic is
now added to the `user-data.sh` for the `upgrade-minimal` operation and
installation of docker, cloudwatch-agent, and curl.

## Testing Done
---
In progress. I'd like to open this up to review while testing.

Co-authored-by: Niek Palm <npalm@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants