Skip to content

Commit

Permalink
fix: add retry logic to runner start script for libicu download (#3748)
Browse files Browse the repository at this point in the history
## Background
---
There seems to be a race condition some where with the user-data script
and the EC2 starting up and locking RPM. This issue is seen elsewhere,
not necessarily with this repo's user-data, see the following:

[Amazon Linux 2023 - issue with installing packages with
cloud-init](https://repost.aws/questions/QU_tj7NQl6ReKoG53zzEqYOw/amazon-linux-2023-issue-with-installing-packages-with-cloud-init)

[dnf/yum both fails while being executed on instance bootstrap on Amazon
Linux
2023](https://repost.aws/questions/QUgNz4VGCFSC2TYekM-6GiDQ/dnf-yum-both-fails-while-being-executed-on-instance-bootstrap-on-amazon-linux-2023)

Also, #3741

## Changes Made
---
Added a loop to retry if the rpm lock file is found which sleeps for 5
seconds then retries again with a total of 5 iterations.

## Testing
---
Spun up an instance and ran the script directly (using sudo the second
time because this was not run via user-data)
<img width="339" alt="image"
src="https://github.com/philips-labs/terraform-aws-github-runner/assets/16331726/33679c00-b092-4900-80fd-838bb6c31409">

---------

Co-authored-by: Niek Palm <npalm@users.noreply.github.com>
  • Loading branch information
jkruse14 and npalm committed Feb 12, 2024
1 parent 5fd1973 commit 1b4597b
Showing 1 changed file with 14 additions and 1 deletion.
15 changes: 14 additions & 1 deletion modules/runners/templates/install-runner.sh
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,20 @@ echo OS: $os_id

# Install libicu on non-ubuntu
if [[ ! "$os_id" =~ ^ubuntu.* ]]; then
dnf install -y libicu
max_attempts=5
attempt_count=0
success=false
while [ $success = false ] && [ $attempt_count -le $max_attempts ]; do
echo "Attempt $attempt_count/$max_attempts: Installing libicu"
dnf install -y libicu
if [ $? -eq 0 ]; then
success=true
else
echo "Failed to install libicu"
attempt_count=$(( attempt_count + 1 ))
sleep 5
fi
done
fi

# Install dependencies for ubuntu
Expand Down

0 comments on commit 1b4597b

Please sign in to comment.