Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Update the batch resizing heuristic in PageProcessor #11901
The yield check in the generated projection tight loop may have negative
This change removes the yield check from the generated projection loop,
Removing the yield check from the projection loop improves the performance
Alternative to #11816.
referenced this pull request
Nov 12, 2018
There were tests that are sensitive to the initial batch size in
Nov 13, 2018
I have run it a few times before opening this PR. Now that you asked I ran it a few more times also with different jmh settings.
With 3 forks, 5 warmup iterations of 10s runs and 5 measurement iterations of 10s runs (the same config used for the results I shared above), I get these two results (~15-25% better):
Then to further make sure that results are stable and get more confidence in the numbers, I increased the fork count to 5, and increased the measurement and warmup runs to both 30s and I got the following result (~8% better):
I have a bunch of comments, but this is looking good. I think we're also going to want a similar change on the filter side, but that can be done later.
Were the results (after increased warmup time):
with your optimization?
@dain comments addressed in a separate commit (will be squashed before merge).
Here are the new u-benchmark results:
Other then the part where timings are converted to millis, looks good.