ic-ref-test: Test stopping state #63
Conversation
|
This is a draft because I found some problems with the |
This allows the test driver to withhold the response to a message, and control when they are released, in order to produce situations with outstanding call contexts. In an ideal world (from our pov), we could instrument and control the system's scheduler this way, but we can't. So instead, we use some tricks. Ideally, the details of this trick are irrelevant to the users of this function (yay, abstraction), and if we find better tricks, we can swap them out easily. We'll see if that holds water. One problem with this approach is that a test failure could mean that the system doesn't pass the test, but it could also mean that the system has a bug that prevents this trick from working, so take care. The current trick is: Create a canister (the "stopper"). Make it its own controller. Tell the canister to stop itself. This call will now hang, because a canister cannot stop itself. We can release the call (producing a reject) by starting the canister again. This has uncovered bugs in `ic-ref` related to the handling of stopped.
nomeata
force-pushed
the
joachim/stopping
branch
from
908cc56
to
ceb30f0
Compare
|
Ok, I patched over the most egregious errors in the reference implementation, so this test now passes. @marcin-dziadus, do you want to check if it works with the replica as well before we merge it. We can add more tests related to outstanding messages later. In particular I would like to test upgrading with outstanding messages, or callbacks to deleted canisters etc. This is something that we don't see a lot in the wild, is not tested here a lot, and maybe not so much in the replica's tests, so it's a bit of blind spot. I think we can make the universal canister cope with callbacks after upgrades, or after deleting/reinstallation, by serializing the callback table. |
|
(this PR was created while traveling in Ghana, on a mobile phone, with external keyboard, via mosh on a rented server. Also a way to work…) |
Sorry for the delay, I've been offline for a while. |
|
Depends: if we think the tests are correct with regard to the spec, and the replica isn't quite in compliance, then we should merge this, use the feature in the replica's CI to mark these tests as known-to-fail, and fix eventually in replica. But if they fail in the replica because the tests are not good (maybe they need to wait a bit in some place or another?) we should fix the tests. How do the tests fail against the replica? |
Thanks for the clarification! Performance counting is currently uncovered in the replica which causes the canister parsing to fail. |
I think it falls under the first type of situations you described. |
|
But this PR is independent of performance counters, isn't it? I'm a bit confused now :-) |
It is :) My point is that running the most recent version of Technically it is possible to check what you asked for, but not without a hassle. |
|
Oh, I see. But it’s kinda unforunate to be blocked here. And we don’t need to: The script that runs So it should be possible to bump Anyways, sounds like we can merge this |
This allows the test driver to withhold the response to a message, and
control when they are released, in order to produce situations with
outstanding call contexts.
In an ideal world (from our pov), we could instrument and control the
system's scheduler this way, but we can't. So instead, we use some tricks.
Ideally, the details of this trick are irrelevant to the users of this
function (yay, abstraction), and if we find better tricks, we can swap them
out easily. We'll see if that holds water.
One problem with this approach is that a test failure could mean that the
system doesn't pass the test, but it could also mean that the system has a
bug that prevents this trick from working, so take care.
The current trick is: Create a canister (the "stopper"). Make it its own
controller. Tell the canister to stop itself. This call will now hang,
because a canister cannot stop itself. We can release the call (producing a
reject) by starting the canister again.
Some tests are added, and others are now using this mechanism.
This has uncovered bugs in
ic-refrelated to the handling of stopped.