Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize ZRANGE replies WITHSCORES in case of integer scores #11779

Merged
merged 7 commits into from Feb 6, 2023

Conversation

filipecosta90
Copy link
Contributor

@filipecosta90 filipecosta90 commented Feb 3, 2023

I started looking at the profile of ZRANGE related commands, like ZREVRANGEBYSCORE due to the benchmark we've added https://github.com/redis/redis-benchmarks-specification/blob/main/redis_benchmarks_specification/test-suites/memtier_benchmark-1key-zrevrangebyscore-256K-elements-pipeline-1.yml .

putting it simple, if we have integer scores on the sorted set we're not using the fastest way to reply by calling d2string which uses double2ll and ll2string when it can, instead of fpconv_dtoa.

To reproduce:

populate sorted sets (cardinality is not really important here -- the score reply type yes) :

redis-cli "ZADD" "zset:10:double_score" "0.306275" "lysbgqqfqw" "0.486004" "mtccjerdon" "0.941626" "jekkafodvk" "0.602656" "nmgxcctxpn" "0.630771" "vyqqkuszzh" "0.406379" "pytrnqdhvs" "0.521814" "oguwnmniig" "0.182854" "gekntrykfh" "0.657658" "nhfnbxqgol" "0.218066" "cgoeihlnei"
redis-cli "ZADD" "zset:10:long_score" "10000000" "lysbgqqfqw" "10000001" "mtccjerdon" "10000002" "jekkafodvk" "10000003" "nmgxcctxpn" "10000004" "vyqqkuszzh" "10000005" "pytrnqdhvs" "10000006" "oguwnmniig" "10000007" "gekntrykfh" "10000008" "nhfnbxqgol" "10000009" "cgoeihlnei"

benchmark:

  1. double replies
# resp2
memtier_benchmark --command="ZREVRANGEBYSCORE zset:10:double_score 100000000 0 LIMIT 1 10 WITHSCORES" --hide-histogram --test-time 60 --pipeline 10 --protocol resp2

# resp3
memtier_benchmark --command="ZREVRANGEBYSCORE zset:10:double_score 100000000 0 LIMIT 1 10 WITHSCORES" --hide-histogram --test-time 60 --pipeline 10 --protocol resp3
  1. long replies
# resp2
memtier_benchmark --command="ZREVRANGEBYSCORE zset:10:long_score 100000000 0 LIMIT 1 10 WITHSCORES" --hide-histogram --test-time 60 --pipeline 10 --protocol resp2

# resp3
memtier_benchmark --command="ZREVRANGEBYSCORE zset:10:long_score 100000000 0 LIMIT 1 10 WITHSCORES" --hide-histogram --test-time 60 --pipeline 10 --protocol resp3

profile info:

Notice that addReplyDouble is taking ~36% of CPU cycles:
image

Impact of this PR in the achievable ops/sec for both double and long scores:

Variation v7.0.8 unstable this PR % change this PR vs v7.0.8 % change this PR vs unstable Note
RESP2 - double score 142997 187233 185812 29.9% -0.8% as confirmed, the overhead of double2ll is neglectible. Notice that this change+unstable vs v7.0.8 still retains a large improvement
RESP3 - double score 142997 184714 183919 28.6% -0.4% as confirmed, the overhead of double2ll is neglectible. Notice that this change+unstable vs v7.0.8 still retains a large improvement
RESP2 - long score 192704 250218 386702 100.7% 54.5% ---
RESP3 - long score 187414 246445 374957 100.1% 52.1% ---

Note that the existing change in unstable missing in 7.0 is #10587

@filipecosta90 filipecosta90 added the action:run-benchmark Triggers the benchmark suite for this Pull Request label Feb 3, 2023
src/t_zset.c Outdated Show resolved Hide resolved
tests/unit/type/zset.tcl Outdated Show resolved Hide resolved
tests/integration/rdb.tcl Show resolved Hide resolved
Copy link
Member

@oranagra oranagra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a reference in the top comment to the PR you refer to (i assume it's #10587, right?)

src/networking.c Outdated Show resolved Hide resolved
src/networking.c Show resolved Hide resolved
src/networking.c Show resolved Hide resolved
@oranagra oranagra added the release-notes indication that this issue needs to be mentioned in the release notes label Feb 6, 2023
@oranagra oranagra merged commit f3c6f9c into redis:unstable Feb 6, 2023
Mixficsol pushed a commit to Mixficsol/redis that referenced this pull request Apr 12, 2023
…1779)

If we have integer scores on the sorted set we're not using the fastest way
to reply by calling `d2string` which uses `double2ll` and `ll2string` when it can,
instead of `fpconv_dtoa`. 

This results by some 50% performance improvement in certain cases of integer
scores for both RESP2 and RESP3, and no apparent impact on double scores.

Co-authored-by: Oran Agra <oran@redislabs.com>
enjoy-binbin pushed a commit to enjoy-binbin/redis that referenced this pull request Jul 31, 2023
…1779)

If we have integer scores on the sorted set we're not using the fastest way
to reply by calling `d2string` which uses `double2ll` and `ll2string` when it can,
instead of `fpconv_dtoa`. 

This results by some 50% performance improvement in certain cases of integer
scores for both RESP2 and RESP3, and no apparent impact on double scores.

Co-authored-by: Oran Agra <oran@redislabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
action:run-benchmark Triggers the benchmark suite for this Pull Request release-notes indication that this issue needs to be mentioned in the release notes
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants