Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Project Management Automation: Avoid gracefully handling error #20009

Merged
merged 1 commit into from Feb 3, 2020

Conversation

@aduth
Copy link
Member

aduth commented Feb 3, 2020

This pull request seeks to change the behavior of our GitHub automation tasks to ensure that they fail loudly, rather than be silently absorbed.

For example, the following action was considered a success, despite the fact that it failed to assign the milestone:

https://github.com/WordPress/gutenberg/runs/423269588?check_suite_focus=true

main: Task addMilestone failed with error: HttpError: Validation Failed: {"resource":"Milestone","code":"already_exists","field":"title"}

We should want these tasks to fail loudly to ensure that they are working properly. This specific tasks has been dysfunctional for quite some time, but was only recently noticed to not be assigning the milestone.

If a task expects to possibly be throwing errors, it should do its own try / catch and deal with the error accordingly. In this case, a duplicate milestone may be "fine" (it was previously anticipated), so the specific step of creating the milestone could be allowed to fail with the duplicate, as long as it still proceeds to add the milestone to the pull request. Currently, this last part was never reached, because it never recovers from the thrown error.

Testing Instructions:

It will likely not be possible to test this, assuming there are no errors in current automation.

That said, there is an error with the milestone assignment automation, which occurs when the next milestone has already been created. This could either be used as verification that the failures are logged loudly, or could be considered a blocker to resolve first before this is merged.

@aduth

This comment has been minimized.

Copy link
Member Author

aduth commented Feb 3, 2020

If a task expects to possibly be throwing errors, it should do its own try / catch and deal with the error accordingly. In this case, a duplicate milestone may be "fine" (it was previously anticipated), so the specific step of creating the milestone could be allowed to fail with the duplicate, as long as it still proceeds to add the milestone to the pull request. Currently, this last part was never reached, because it never recovers from the thrown error.

See also: #20011

@aduth

This comment has been minimized.

Copy link
Member Author

aduth commented Feb 3, 2020

I'm merging this first, before #20011, anticipating that the action will fail with an error, but doing so intentionally to verify the expected behavior that the task error is not silently absorbed.

@aduth aduth merged commit 2e00995 into master Feb 3, 2020
2 checks passed
2 checks passed
pull-request-automation
Details
Travis CI - Pull Request Build Passed
Details
@aduth aduth deleted the update/pm-automation-fail-loudly branch Feb 3, 2020
@aduth

This comment has been minimized.

Copy link
Member Author

aduth commented Feb 3, 2020

Interesting! The process failed, but the task itself was still reported as a success:

https://github.com/WordPress/gutenberg/runs/423870820?check_suite_focus=true

(node:2810) UnhandledPromiseRejectionWarning: HttpError: Validation Failed: {"resource":"Milestone","code":"already_exists","field":"title"}
at /home/runner/work/gutenberg/gutenberg/packages/project-management-automation/node_modules/@octokit/request/dist-node/index.js:66:23
at processTicksAndRejections (internal/process/task_queues.js:93:5)
at async addMilestone (/home/runner/work/gutenberg/gutenberg/packages/project-management-automation/lib/add-milestone.js:88:2)
at async main (/home/runner/work/gutenberg/gutenberg/packages/project-management-automation/lib/index.js:52:4)

I'll follow-up to investigate whether there are documented options for signalling that the the action failed.

Also because at some point in the future, we'd need to handle the failure explicitly anyways:

(node:2810) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

@aduth

This comment has been minimized.

Copy link
Member Author

aduth commented Feb 3, 2020

There appear to be two options, based on the documentation.

Using the setFailed message from the @actions/core package:

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/creating-a-javascript-action#write-the-action-code

Or setting the exit code of the process:

https://help.github.com/en/actions/automating-your-workflow-with-github-actions/development-tools-for-github-actions#setting-an-actions-status-using-exit-codes

I might have thought the uncaught error would already be causing a non-zero status code, but apparently not?

@epiqueras

This comment has been minimized.

Copy link
Contributor

epiqueras commented Feb 3, 2020

Unhandled promise rejections

Don't do that. We need to do it explicitly.

@aduth

This comment has been minimized.

Copy link
Member Author

aduth commented Feb 3, 2020

Follow-up at #20012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.