Nursery: don't act as a checkpoint when not running anything #1696
Conversation
Here's the code that does that, at line 923: def aborted(raise_cancel):
self._add_exc(capture(raise_cancel).error)...I was going to say we could delete this line and simply ignore cancellation here, but that's not quite right. We do want to ignore |
|
I left a comment on #1457: I think we shouldn't drop the schedule point in either branch, only the cancellation point; and we shouldn't drop the cancellation point if the nursery didn't run any tasks, so that The test is failing because it relies on nursery exit being a schedule point. |
Previously, a Nursery could raise Cancelled even if all its children had completed successfully. This is undesirable, as discussed in python-trio#1457, so now a Nursery will shield itself from cancels in __aexit__. A Nursery with children can still be cancelled fine: if any of its children raise Cancelled, then the whole Nursery will raise Cancelled. If none of the Nursery's children raise Cancelled, then the Nursery will never raise Cancelled. As another consequence, a Nursery which never had any child tasks will never raise Cancelled. As yet another consequence, since an internal Nursery is used in the implementation of Nursery.start(), if start() is cancelled, but the task it's starting does not raise Cancelled, start() won't raise Cancelled either.
trio/_core/tests/test_run.py
Outdated
| # but if the task does not execute any checkpoints, and exits, then start() | ||
| # doesn't raise Cancelled, since the task completed successfully. | ||
| async def started_with_no_checkpoint(task_status=_core.TASK_STATUS_IGNORED): | ||
| task_status.started("hi") | ||
|
|
||
| async with _core.open_nursery() as nursery: | ||
| with _core.CancelScope() as cs: | ||
| cs.cancel() | ||
| await nursery.start(started_with_no_checkpoint) | ||
|
|
There was a problem hiding this comment.
Some more implications here. This is because a Nursery is used in the implementation of start().
I'd argue that this is correct.
|
Also, what do you think about making assert_yields_or_not public? I just needed it for the test that a nursery is a schedule point but not a cancel point. Maybe that's something we should discourage? |
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #1696 +/- ##
==========================================
+ Coverage 99.03% 99.14% +0.10%
==========================================
Files 115 115
Lines 17433 17456 +23
Branches 3107 3110 +3
==========================================
+ Hits 17265 17306 +41
+ Misses 122 104 -18
Partials 46 46
|
I actually just ran into this issue again while working on #1805 on stream, so I did some tweaks to land this quicker. And one of those tweaks was to revert the |
There's no easy way to work around it, and it's not worth it anyway since almost no-one uses ancient 3.6.x point releases. (It was fixed in 3.6.2.)
| When exiting a nursery block, the parent task always waits for child | ||
| tasks to exit. This wait cannot be cancelled. However, previously, if | ||
| you tried to cancel it, it *would* inject a `Cancelled` exception, | ||
| even though it wasn't cancelled. Most users probably never noticed |
There was a problem hiding this comment.
This wait cannot be cancelled. However, previously, if
you tried to cancel it, it would inject a Cancelled exception,
even though it wasn't cancelled.
So this means the Cancelled is still injected but now is ignored via the new isinstance check?
Also if the following is true:
This wait cannot be cancelled. However, previously, if
you tried to cancel it, it would inject a Cancelled exception,
even though it wasn't cancelled.
trio/_core/tests/test_run.py
Outdated
| summary = {} | ||
| for exc in multi_exc.exceptions: | ||
| summary.setdefault(type(exc), 0) | ||
| summary[type(exc)] += 1 | ||
| assert summary == {_core.Cancelled: 4, KeyError: 1} | ||
| assert summary == {_core.Cancelled: 3, KeyError: 1} |
There was a problem hiding this comment.
Clarifying the cancelled count in a comment might be clearer here for those unfamiliar with the recent change?
Something like we only receive cancelleds from subtasks not from the nursery itself?
trio/_core/tests/test_run.py
Outdated
| with _core.CancelScope() as cs: | ||
| cs.cancel() | ||
| async with _core.open_nursery() as nursery: | ||
| nursery.start_soon(noop_with_no_checkpoint) |
There was a problem hiding this comment.
I presume it's out of scope (and already covered in other tests) but normally doing this cancellation before opening the nursery results in cancellation at the first checkpoint right?
trio/_core/tests/test_run.py
Outdated
|
|
||
| async with _core.open_nursery() as nursery: | ||
| with _core.CancelScope() as cs: | ||
| cs.cancel() |
There was a problem hiding this comment.
In the case where you cancel from inside the nursery after having started tasks with checkpoints, won't counts of expected cancels change as well or is that entirely covered by the one test change in test_run.py?
|
The changes look fine to me. Excited for this to be fixed! |
# Conflicts: # trio/_core/_run.py # trio/_core/_tests/test_run.py
|
The merge was not as painful as one might expect. The two new commits are nearly trivial fixes responsive to recent updates to the test suite. |
|
Hey @catern, it looks like that was the first time we merged one of your PRs! Thanks so much! 🎉 🎂 If you want to keep contributing, we'd love to have you. So, I just sent you an invitation to join the python-trio organization on Github! If you accept, then here's what will happen:
If you want to read more, here's the relevant section in our contributing guide. Alternatively, you're free to decline or ignore the invitation. You'll still be able to contribute as much or as little as you like, and I won't hassle you about joining again. But if you ever change your mind, just let us know and we'll send another invitation. We'd love to have you, but more importantly we want you to do whatever's best for you. If you have any questions, well... I am just a humble Python script, so I probably can't help. But please do post a comment here, or in our chat, or on our forum, whatever's easiest, and someone will help you out! |
Previously, a Nursery would checkpoint on exit, regardless of whether
it was running anything. This is a bit annoying, see #1457, so let's
change it to not checkpoint on exit. Now it won't be a checkpoint at
all if there's no tasks running inside.
However, this doesn't fully solve #1457, because we'll still inject a
cancel even if none of the child tasks raise Cancelled.
This seems to break
trio/_core/tests/test_asyncgen.py::test_last_minute_gc_edge_case
such that it no longer ever hits the edge case.
That seems like a pretty complicated test, so maybe @oremanj can say
better why exactly it's breaking with this change.