Make precomputing fail more usefully #7994
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
stuhood
reviewed
Jul 2, 2019
| @@ -102,6 +102,21 @@ def exit_code(self): | |||
| return self._exit_code | |||
|
|
|||
|
|
|||
| class _PantsProductPrecomputeFailed(Exception): | |||
There was a problem hiding this comment.
So, at a fundamental level, there is no need to "precompute" or "warm" anything anymore. So we should actually be able to delete the warming codepath entirely.
stuhood
approved these changes
Jul 2, 2019
|
|
||
| # TODO This can probably be moved to the try:except block in DaemonPantsRunner.run() function |
There was a problem hiding this comment.
Linking to a ticket that explains why, or expanding the description here would be good. There is definitely dead code in here.
There was a problem hiding this comment.
Opened #8002, I hope it's enough for most people to pick up, but would definitely love feedback to make it more approachable.
|
Oh no, just noticed the needs cherry-pick label 😢 this can't use Py3 features like f-strings if this needs to be included in 1.17.x. |
ity
approved these changes
Jul 2, 2019
blorente
force-pushed
the
blorente/make-precomputing-fail-properly
branch
from
dce73e1
to
4470df9
Compare
|
I added some unit tests for these specific cases, if CI is green I will merge tomorrow. |
illicitonion
approved these changes
Jul 5, 2019
blorente
added a commit
to blorente/pants
that referenced
this pull request
Jul 5, 2019
**Problem**
DaemonPantsRunner precomputes the v2 graph products at creation time (before run). If that raised an exception, it would store a return code of 1, run normally, and then exit with 1 at the end:
class DaemonPantsRunner:
...
def run(self):
....
else:
self._exiter.exit(self.**exit_code** if self.exit_code else PANTS_SUCCEEDED_EXIT_CODE)
This hints that the behaviour that we want is to actually fail the run if the precomputation fails. If that is the case, we would like to fail more usefully, with a real exception.
**Solution**
Create a wrapper exception type that wraps whatever error happened in the precomputation. Then, in DPR.run(), re-raise this deferred exception immediately after setting up logging and stdio channels.
**Result**
What before seemed to succeed and returned 1, should now return a proper exception.
blorente
added a commit
to blorente/pants
that referenced
this pull request
Jul 8, 2019
It introduced failures in the nightly cron (pantsbuild#8023), and a bunch more errors when we tested it on Twitter internal targets.
blorente
added a commit
that referenced
this pull request
Jul 10, 2019
illicitonion
pushed a commit
that referenced
this pull request
Jul 10, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
DaemonPantsRunnerprecomputes the v2 graph products at creation time (beforerun). If that raised an exception, it would store a return code of 1, run normally, and then exit with 1 at the end:This hints that the behaviour that we want is to actually fail the run if the precomputation fails. If that is the case, we would like to fail more usefully, with a real exception.
Solution
Create a wrapper exception type that wraps whatever error happened in the precomputation. Then, in
DPR.run(), re-raise this deferred exception immediately after setting up logging and stdio channels.Result
What before seemed to succeed and returned 1, should now return a proper exception.