KAFKA-4851: only search available segments during Segments.segments(from, to) #2645
Conversation
|
@guozhangwang @enothereska @mjsax - found this today when someone on the user list reported strange behaviour with Session Windows. Found it performed terribly with caching enabled, but performed well with it turned off. Eventually narrowed it down to calling |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
| final long segFrom = segmentId(Math.max(0L, timeFrom)); | ||
| final long segTo = segmentId(Math.min(currentSegmentId * segmentInterval, Math.max(0, timeTo))); | ||
| final long segFrom = Math.max(minSegmentId, segmentId(Math.max(0L, timeFrom))); | ||
| final long segTo = Math.min(currentSegmentId, segmentId(Math.min(currentSegmentId * segmentInterval, Math.max(0, timeTo)))); |
There was a problem hiding this comment.
nit: shall we rename currentSegmentId to sth. like latestSegmentId or maxSegmentId?
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
|
Refer to this link for build results (access rights to CI server needed): |
|
Jenkins failures seem to be related to https://issues.apache.org/jira/browse/KAFKA-4841. There is a fix ongoing: #2641 |
…rom, to) restrict the locating of segments in `Segments#segments(..)` to only the segments that are currently available, i.e., rather than searching the hashmap for many segments that don't exist. Author: Damian Guy <damian.guy@gmail.com> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes #2645 from dguy/session-windows-testing (cherry picked from commit 146edd5) Signed-off-by: Guozhang Wang <wangguoz@gmail.com>
|
Merged to trunk and cherry-picked to 0.10.2. |
restrict the locating of segments in
Segments#segments(..)to only the segments that are currently available, i.e., rather than searching the hashmap for many segments that don't exist.