MSQ extension: Fix over-capacity write in ScanQueryFrameProcessor.#13036
MSQ extension: Fix over-capacity write in ScanQueryFrameProcessor.#13036abhishekagarwal87 merged 2 commits intoapache:masterfrom
Conversation
Frame processors are meant to write only one output frame per cycle. The ScanQueryFrameProcessor would write two when reading from a channel if the input frame cursor cycled and then the output frame filled up while reading from the next frame. This patch fixes the bug, and adds a test. It also makes some adjustments to the processor code in order to make it easier to test.
gianm
added
Bug
Area - MSQ
|
This part is the fix: https://github.com/apache/druid/pull/13036/files#diff-9b472caafa614f6c7b93fc27e09e8192b1bd6b665857ed0e5946f73035ac633cL206-R207 The rest is refactoring to make it possible to write the test. |
cryptoe
left a comment
cryptoe
left a comment
There was a problem hiding this comment.
LGTM.
It would be cool to understand how you debugged this also.
vogievetsky
left a comment
vogievetsky
left a comment
There was a problem hiding this comment.
I saw Gian write the fix :-)
I ran into an error "Channel has no capacity" from BlockingQueueFrameChannel in a test cluster. The stack trace indicated it was a frame write from ScanQueryFrameProcessor |
…13036) * MSQ extension: Fix over-capacity write in ScanQueryFrameProcessor. Frame processors are meant to write only one output frame per cycle. The ScanQueryFrameProcessor would write two when reading from a channel if the input frame cursor cycled and then the output frame filled up while reading from the next frame. This patch fixes the bug, and adds a test. It also makes some adjustments to the processor code in order to make it easier to test. * Add license header.
4 participants
Frame processors are meant to write only one output frame per cycle. The ScanQueryFrameProcessor would write two when reading from a channel if the input frame cursor cycled and then the output frame filled up while reading from the next frame.
This patch fixes the bug, and adds a test. It also makes some adjustments to the processor code in order to make it easier to test.