KAFKA-2989: system tests should verify partitions consumed after rebalancing #702
Conversation
| @@ -123,7 +123,7 @@ class VerifiableConsumer(BackgroundThreadService): | |||
| logs = { | |||
| "verifiable_consumer_stdout": { | |||
| "path": STDOUT_CAPTURE, | |||
| "collect_default": False}, | |||
| "collect_default": True}, | |||
There was a problem hiding this comment.
Was this intentional or just for debugging? Verifiable consumer can generate a huge amount of output, so its pretty risky to save it, especially if we don't have a way to ensure it is at least compressed first. We definitely always need to grab it to do verification, but not necessarily save it by default with the output (which would happen even if the test was successful).
There was a problem hiding this comment.
It was intentional since it helps debugging quite a bit. The verifiable consumer only logs the start and end offsets of the batches of records it consumes, so the size is pretty reasonable. At the same time, it's probably unnecessary since I already increased offset logging in that other ticket.
|
@hachikuji A bunch of small questions, but overall this looks like a nice cleanup and extension. All the affected tests are passing cleanly for me here. |
|
@ewencp Ready for another look when you get a chance. |
|
LGTM |
No description provided.