integration-cli: Better test cleanup with defer #10909
Conversation
ahmetb
changed the title
|
I wonder if it makes sense to create a generic cleanup() func that will do things like erase all images and containers and then make all tests start with a defer cleanup(). Then when we want to add more cleanup its only one spot that needs to be modified. |
|
@duglin pretty much right, but I'll leave this to another PR as this makes the job easy for that. |
|
Also, there is some predefined images for us - |
| @@ -304,6 +305,8 @@ func TestPsListContainersFilterStatus(t *testing.T) { | |||
| // FIXME: this should test paused, but it makes things hang and its wonky | |||
| // this is because paused containers can't be controlled by signals | |||
| deleteAllContainers() | |||
There was a problem hiding this comment.
@LK4D4 I'm not sure, this guy might have a reason to ask for preemptive deletion of call containers 'cause it would absolutely fail in case of a leak and it'd be hard to trace which test case has leaked a container.
There was a problem hiding this comment.
I can say this about all tests. We should fix our suite, not hack it.
There was a problem hiding this comment.
Yes. Though, this PR is not a hack, it's actually removing ~250 lines of potentially dead code. That deleteAllContainers() without defer is totally harmless and this specific test case is more vulnerable to leftover contention from other tests, that's why I am keeping the status quo.
There was a problem hiding this comment.
But that deleteAllContainers a hack. And it used only for this test, though another Ps tests is the same vulnerable to leaked containers, so I don't see a point for keeping it.
|
@ahmetalpbalkan Thank you so much. |
|
@LK4D4 my pleasure. For deleteImages, we're doing a cleanup at the end and tests requiring an empty image list probably clean the env before they start. However we may care more about cleaning containers between test methods might be important as we sometimes use a hardcoded container name or port number. It'll increase execution time per test case. Also the CI suite has a cleanup, so that serves the same purpose. I'm now proposing a "webserver" image that shouldn't also be deleted like "busybox', in the meanwhile (#10893) so I am not really in for creating a deleteAllImagesExceptSome() route. |
|
So, we're essentially saying we can't run any of our tests in parallel since each test might blow away resources in use by the others instead of just resources it individually creates? |
|
It's less error-prone overall, but also a lot less flexible IMO. 😕 |
|
@tianon I don't understand what you saying :) It was always like this. |
|
@tianon we can add some check like DCO validaton and force every test to begin with "defer cleanup()" in the future. |
|
Not all tests are -- some are really very good about only cleaning up after
themselves, and using test-unique names appropriately to ensure that this
is the case.
|
|
I'm not sure that we can run some tests parallel at all. Those tests which requires some clean state, which can't be cleaned by hands(events for example) should use separate daemon. |
This fixes a few misuses of `deleteAllContainers()` cleanup method in integration-cli suite by moving call to the beginning of the method and guaranteeing their execution (including panics) with `defer`s. Also added some forgotten cleanup calls while I'm at it. Signed-off-by: Ahmet Alp Balkan <ahmetalpbalkan@gmail.com>
|
LGTM |
1 similar comment
|
LGTM |
integration-cli: Better test cleanup with defer
This fixes a few misuses of
deleteAllContainers()cleanupmethod in integration-cli suite by moving call to the
beginning of the method and guaranteeing their execution
(including panics) with
defers. Most calls were not actuallyhappening in case of t.Fatal.
Also added some forgotten cleanup calls while I'm at it.
Signed-off-by: Ahmet Alp Balkan ahmetalpbalkan@gmail.com
cc: @unclejack @duglin @tianon @jfrazelle