Skip to content

Retry deployment after 10 seconds 3 Times #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 7, 2025

Conversation

0tuedon
Copy link
Collaborator

@0tuedon 0tuedon commented Mar 6, 2025

From my research, the error mainly happens with Railway (sometimes the authentication skips), and then it fails. I’m adding a retry mechanism that will try 3 times. I believe it will work because when I manually restart the workflow, it usually passes the second time. If this doesn’t work, we can easily undo this change.

cc @satsie , @jrakibi

Copy link
Member

@satsie satsie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good to me! Let's give it a go and see if the failures go away 👍

@0tuedon 0tuedon merged commit dbea407 into bitcoin-dev-project:main Mar 7, 2025
@jrakibi
Copy link
Contributor

jrakibi commented Mar 7, 2025

cross posting this comment here:

I think the the current script will exit with success even if all three attempts fail (because the last command executed is either sleep or echo both of which return a success status). This means the job might incorrectly report success even if the deployment fails.



We can add a check like this at the end with exit 1 command


- name: Deploy
  run: |  # This script retries the deployment 3 times before failing
    n=0
    until [ "$n" -ge 3 ]
    do
      railway up --service public-inbox && break
      n=$((n+1))
      echo "Retrying deployment... attempt $n"
      sleep 15  # Increased delay between retries
    done
    
    # Check if all retries failed
    if [ "$n" -eq 3 ]; then
      echo "All deployment attempts failed"
      exit 1 
    fi
  env:
    RAILWAY_TOKEN: ${{ secrets.RAILWAY_TOKEN }}

We can do this later. Let's first see what happens with the current changes merged and check the logs, even if it reports a success status

@satsie
Copy link
Member

satsie commented Mar 7, 2025

oh my! good catch @jrakibi!!! This should get addressed in a follow up PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants