Description
Problem Description
The app may crash if you misspell something in the config, with the check run being stuck on the PR without any clues reported back to the PR on what's going on.
What is actually happening
Example from the current version is that the username is forced to lowercase in the collaborators plugin.
We had an example of a PR where the collaborator was specified using name: githubuser instead of the correct syntax username: githubuser. The PR checks got stuck on "Queued check run" but never reported back to the PR.
Only after pushing more commits to the same PR we got the following error in the lambda logs:
It would of course be good to add validation specific to every plugin, but there should still be proper handling of unexpected errors so the check runs don't get stuck.
What is the expected behavior
- The validation step should always return an error and not crash
- Queued check runs should not get stuck because of config syntax errors
- For unexpected errors, the check run should at minimum complete as failed, preferably with the error message included to avoid log diving.
Error output, if available
Context
Are you using the hosted instance of probot/settings or running your own?
If running your own instance, are you using it with github.com or GitHub Enterprise?
Version of probot/settings
2.0.21-rc.7