Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

'All Stats' output contains zeros for all the columns except 'Latency' #128

Closed
speddam opened this issue May 21, 2020 · 4 comments
Closed

Comments

@speddam
Copy link

speddam commented May 21, 2020

Hi,

The latest memtier_benchmark shows the 'All Stats' output containing all zeros for all the columns except 'Latency', like this:

$ ./memtier_benchmark --clients=50 --threads=4 --requests=10000 --data-size-range=32-1000 --pipeline=100 --ratio=5:5
...
ALL STATS
=========================================================================
Type         Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec 
-------------------------------------------------------------------------
Sets            0.00          ---          ---     19.71200         0.00 
Gets            0.00         0.00         0.00     19.73500         0.00 
Waits           0.00          ---          ---      0.00000          --- 
Totals          0.00         0.00         0.00     19.72300         0.00
...

I have done the git bisect and it pointed to this commit [0]:

$ git bisect bad
bec9cc46079476c8414ab7104516c5a9c966b90f is the first bad commit
commit bec9cc46079476c8414ab7104516c5a9c966b90f
Author: YaacovHazan <31382944+YaacovHazan@users.noreply.github.com>
Date:   Wed Dec 11 17:02:15 2019 +0200

    fix issue #94 cluster mode stuck (#106)

    add missing bufferevent_disable for connections that
    complete to send new requests

:100644 100644 b9fe31c3568e172369897c0eea0111138dcc5e8b 6fbcd856130c8709f8786c97e6b38ad101ca342c M	shard_connection.cpp

I'm not familiar with the code base, but by commenting out the bufferevent_disable() call, I see that the output now shows all the columns:

ALL STATS
=========================================================================
Type         Ops/sec     Hits/sec   Misses/sec      Latency       KB/sec 
-------------------------------------------------------------------------
Sets       509744.79          ---          ---     19.52900    281230.68 
Gets       509744.79       509.74    509235.05     19.53200     20145.57 
Waits           0.00          ---          ---      0.00000          --- 
Totals    1019489.58       509.74    509235.05     19.53000    301376.25

Regards,
Subhash

[0] bec9cc4?diff=unified#commitcomment-38744660

@rohitjoshi
Copy link

I have the same issue and by commenting out the line, it works fine.

@@ -473,7 +473,7 @@ void shard_connection::fill_pipeline(void)
         if (m_conns_manager->hold_pipeline(m_id)) {
             // if we are still connected, disable any event
             if (get_connection_state() == conn_connected) {
-                bufferevent_disable(m_bev, EV_WRITE|EV_READ);
+                //bufferevent_disable(m_bev, EV_WRITE|EV_READ);
             }

             break;

YaacovHazan added a commit to YaacovHazan/memtier_benchmark that referenced this issue Jun 24, 2020
the issue relates to the way each connection disable its events
move that code to one place and handle it separately for read/write
@yaacovhazan-Redislabs
Copy link
Contributor

thank you @speddam and @rohitjoshi for raising the issue and narrow it to the specific line where the issue started... I created PR #130, that should fix it.

yaacovhazan-Redislabs pushed a commit that referenced this issue Jun 29, 2020
the issue relates to the way each connection disable its events
move that code to one place and handle it separately for read/write
@yossigo
Copy link
Contributor

yossigo commented Aug 16, 2020

Closing this as I assume it's fixed, please re-open if you still experience a problem.

@godjoem
Copy link

godjoem commented Aug 23, 2024

Hi, I have a similar issue when running memtier_benchmark with EVAL command in cluster mode. (It's fine when running on a standalone Redis)
The command:

memtier_benchmark -s 127.0.0.1 -p 6380 -c 10 -n 100 --cluster-mode \
  --key-prefix={account}: --key-minimum=1 --key-maximum=10000 \
  --command='EVAL "return redis.call(\"SET\", KEYS[1], ARGV[1])" 1 __key__ __data__' \
  --command-key-pattern=P

Output:

ALL STATS
============================================================================================================================
Type         Ops/sec    MOVED/sec      ASK/sec    Avg. Latency     p50 Latency     p99 Latency    p100 Latency       KB/sec 
----------------------------------------------------------------------------------------------------------------------------
Evals           0.00         0.00         0.00         0.54092         0.39100         3.05500        12.41500         0.00 
Totals          0.00         0.00         0.00         0.54092         0.39100         3.05500        12.41500         0.00 

I use memtier_benchmark 2.1.1 and run Redis cluster in docker containers (latest Redis image).

The output is fine when running the command speddam uses
memtier_benchmark --clients=50 --threads=4 --requests=10000 --data-size-range=32-1000 --pipeline=100 --ratio=5:5 -p 6380 --cluster-mode

I found that this problem occurs when I execute the EVAL command on a Redis cluster.

May you guys kindly help? @YaacovHazan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants