Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Don't cancel workflow on errors #139

Merged
merged 5 commits into from Oct 31, 2022
Merged

fix: Don't cancel workflow on errors #139

merged 5 commits into from Oct 31, 2022

Commits on Oct 31, 2022

  1. fix: Don't cancel workflow on errors (#72) 

    (a) Stop idle runners after ten minutes to prevent resource waste due to stolen runners or user-cancelled workflows
(b) Retry stopping runners on error until they're stopped to avoid having to cancel the workflow

The following use cases were tested. After each we need to make sure no dead runners remain registered (there is a limit that will prevent new runners from registering), and that runner resources don't stay and waste money.

1. Lambda function that times out before a job is accepted (job cancelled, or stolen by a previous runner)
2. Lambda function that times out after a job is accepted (timeout too short, or job too long)
3. All providers: simulate runner not accepting any job in 5 minutes by executing step function manually (job cancelled, or stolen by previous runner)

This doesn't fix the problem where a failed runner execution will leave the job waiting. This can still lead to stealing jobs, but at least it won't leave long-running resources behind waiting for a job and wasting money.
    @kichik
    kichik committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    db3b0d1 View commit details
    Browse the repository at this point in the history

  2. Update snapshot

    @kichik
    kichik committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    bf82b7d View commit details
    Browse the repository at this point in the history

  3. Projen hates Windows :(

    @kichik
    kichik committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    e3ba68f View commit details
    Browse the repository at this point in the history

  4. Delete node_modules and snapshot AGAIN

    @kichik
    kichik committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    9dacbbc View commit details
    Browse the repository at this point in the history

  5. chore: self mutation 

    Signed-off-by: github-actions <github-actions@github.com>
    github-actions committed Oct 31, 2022
    Configuration menu
    Copy the full SHA
    2b7eda9 View commit details
    Browse the repository at this point in the history