Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ROCKSDB_PERF_CONTEXT doesn't count the second table in join #1018

Closed
spetrunia opened this issue Apr 25, 2019 · 1 comment
Closed

ROCKSDB_PERF_CONTEXT doesn't count the second table in join #1018

spetrunia opened this issue Apr 25, 2019 · 1 comment

Comments

@spetrunia
Copy link
Contributor

I am doing performance analysis of a join query. Access to one of the joined table is counted in I_S.ROCKSDB_PERF_CONTEXT. Access to the second one is not. This is very confusing.

Steps to reproduce:
Step 1 : create the test dataset and set rocksdb_perf_context_level=3:

insert into ten values (0),(1),(2),(3),(4),(5),(6),(7),(8),(9);

create table one_k(a int);
insert into one_k select A.a + B.a* 10 + C.a * 100 from ten A, ten B, ten C;

set rocksdb_max_row_locks=1000*1000*1000;
set rocksdb_commit_in_the_middle=1;

create table t20 (pk int primary key, a int) engine=rocksdb;
create table t21 (pk int primary key, a int) engine=rocksdb;

SET rocksdb_perf_context_level=3;
insert into t20 select A.a+1000*B.a, A.a+1000*B.a from one_k A, ten B;
insert into t21 select A.a+1000*B.a, A.a+1000*B.a from one_k A, ten B;

set global rocksdb_force_flush_memtable_now=1;

Step2: save Perf Context counter values

create temporary table run1_start as 
select * from information_schema.ROCKSDB_PERF_CONTEXT
where table_name in ('t20', 't21');
# This shows: Query OK, 114 rows affected (0.00 sec)

Step3: run the query. Let's check the EXPLAIN first.

set join_buffer_size=500*1024*1024;
explain
select count(*) from t20,t21 where t20.a=t21.a;
+----+-------------+-------+------+---------------+------+---------+------+-------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key  | key_len | ref  | rows  | Extra                                              |
+----+-------------+-------+------+---------------+------+---------+------+-------+----------------------------------------------------+
|  1 | SIMPLE      | t20   | ALL  | NULL          | NULL | NULL    | NULL | 10000 | NULL                                               |
|  1 | SIMPLE      | t21   | ALL  | NULL          | NULL | NULL    | NULL | 10000 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+------+---------------+------+---------+------+-------+----------------------------------------------------+

Ok, it's going to do a full table scan on both tables. Depending on the buffer size, the second table can be scanned multiple times.

Run the query:

MySQL [test2]> select count(*) from t20,t21 where t20.a=t21.a;
+----------+
| count(*) |
+----------+
|    10000 |
+----------+
1 row in set (6.17 sec)

Step4: Save the counter values after query:

MySQL [test2]> create temporary table run1_end as 
    -> select * from information_schema.ROCKSDB_PERF_CONTEXT
    -> where table_name in ('t20', 't21');
Query OK, 114 rows affected (0.00 sec)
Records: 114  Duplicates: 0  Warnings: 0

Step 5: check counter increments. First, lets' check table t20:

select 
   tmp2.table_name,
   tmp2.stat_type,
   tmp2.value - tmp1.value
from run1_start tmp1, run1_end tmp2
where
   tmp2.stat_type= tmp1.stat_type and
   tmp2.table_name=tmp1.table_name and 
   tmp2.table_schema=tmp1.table_schema
   and tmp2.table_name='t20'
order by 
stat_type;
+------------+---------------------------------+-------------------------+
| table_name | stat_type                       | tmp2.value - tmp1.value |
+------------+---------------------------------+-------------------------+
| t20        | BLOCK_CACHE_HIT_COUNT           |                       7 |
| t20        | BLOCK_CHECKSUM_TIME             |                  286975 |
| t20        | BLOCK_DECOMPRESS_TIME           |                    3625 |
| t20        | BLOCK_READ_BYTE                 |                  357129 |
| t20        | BLOCK_READ_COUNT                |                      88 |
| t20        | BLOCK_READ_TIME                 |                  214990 |
| t20        | BLOCK_SEEK_NANOS                |                   10192 |
| t20        | BLOOM_MEMTABLE_HIT_COUNT        |                       0 |
| t20        | BLOOM_MEMTABLE_MISS_COUNT       |                       0 |

There are some non-zero increments. No problem so far.

Now, let's check t21:

select 
   tmp2.table_name,
   tmp2.stat_type,
   tmp2.value - tmp1.value
from run1_start tmp1, run1_end tmp2
where
   tmp2.stat_type= tmp1.stat_type and
   tmp2.table_name=tmp1.table_name and 
   tmp2.table_schema=tmp1.table_schema
   and tmp2.table_name='t21'
order by 
stat_type;
+------------+---------------------------------+-------------------------+
| table_name | stat_type                       | tmp2.value - tmp1.value |
+------------+---------------------------------+-------------------------+
| t21        | BLOCK_CACHE_HIT_COUNT           |                       0 |
| t21        | BLOCK_CHECKSUM_TIME             |                       0 |
| t21        | BLOCK_DECOMPRESS_TIME           |                       0 |
| t21        | BLOCK_READ_BYTE                 |                       0 |
| t21        | BLOCK_READ_COUNT                |                       0 |
| t21        | BLOCK_READ_TIME                 |                       0 |
| t21        | BLOCK_SEEK_NANOS                |                       0 |
| t21        | BLOOM_MEMTABLE_HIT_COUNT        |                       0 |
| t21        | BLOOM_MEMTABLE_MISS_COUNT       |                       0 |
| t21        | BLOOM_SST_HIT_COUNT             |                       0 |
| t21        | BLOOM_SST_MISS_COUNT            |                       0 |
| t21        | DB_CONDITION_WAIT_NANOS         |                       0 |

ALL counters had zero increments. I think this is wrong.

@spetrunia
Copy link
Contributor Author

This is actually intentional. ha_rocksdb.cc has this:

      to maintain perf_context on a per table basis. Therefore, roll all
      perf_context data into the first table used in a query. This works well
      for single table queries and is probably good enough for queries that hit
      multiple tables.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant