rocksdb: crash & corruption, charsets, indexed cols #784
Assignees
Comments
|
and on different tables definition, there is no crash, but the table is just reported as "corrupt", hence the 'check table' in there. If you need more testcases, I can supply them later.. |
|
The problem here is that when we encode varchar, we use a space padded encoding that puts in unpack info how much many trailing spaces the column contains, so that we can restore the original value for index-only scans. We do this even for collations that we no hope of unpacking, and when we read the row, we call To avoid a format change, the best solution is probably just to skip those bytes during decode. |
facebook-github-bot
pushed a commit
that referenced
this issue
Dec 23, 2019
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes #784 Closes #785 Differential Revision: D6699483 fbshipit-source-id: d0e042c
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 12, 2020
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 fbshipit-source-id: d0e042c
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Sep 9, 2020
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 fbshipit-source-id: d0e042c
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Sep 16, 2020
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 fbshipit-source-id: d0e042c
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Oct 5, 2020
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 fbshipit-source-id: d0e042c
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Nov 11, 2020
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 fbshipit-source-id: d0e042c
facebook-github-bot
pushed a commit
that referenced
this issue
Mar 8, 2021
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes #784 Closes #785 Differential Revision: D6699483 (f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 16, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 17, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 30, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 30, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Aug 31, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Sep 1, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Sep 2, 2021
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483 (facebook@f3d3695) fbshipit-source-id: 7707a3173f7
hermanlee
pushed a commit
that referenced
this issue
Jan 10, 2022
Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes #784 Closes #785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jan 17, 2022
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jan 17, 2022
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
May 20, 2022
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
May 23, 2022
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
laurynas-biveinis
pushed a commit
to laurynas-biveinis/mysql-5.6
that referenced
this issue
Aug 11, 2022
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Mar 25, 2023
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Mar 28, 2023
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jun 1, 2023
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
inikep
pushed a commit
to inikep/mysql-5.6
that referenced
this issue
Jun 14, 2023
…acebook#785) Summary: We always generate unpack info bytes that indicate how much trailing space we need, even when the field can never be unpacked, because it is not implemented. When skipping these fields however, we only skip the key portion, and not the value portion which corrupts data for code that reads unpack_info downstream. Fix by skipping these bytes. Closes facebook#784 Closes facebook#785 Differential Revision: D6699483
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reported to Percona Server and confirmed upstream fb-mysql-5.6.35 at commit 245589f
To reproduce :
set sql_mode ='';
drop table if exists t;
create table t (
h varchar(31) character set utf8 collate utf8_bin not null,
i varchar(19) collate latin1_bin not null,
primary key(i),
key(h)
) engine=rocksdb;
insert into t(i,h) values('a','b');
check table t;
alter table t modify h varchar(31) character set cp1257 collate cp1257_bin not null;
check table t;
Call stack :
#0 0x00007ffff5f53c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff5f57028 in __GI_abort () at abort.c:89
#2 0x00007ffff5f4cbf6 in __assert_fail_base (fmt=0x7ffff60a1018 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x1d6fcd5 "netbuf != nullptr",
file=file@entry=0x1d6fc78 "/ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/././rdb_buff.h", line=line@entry=121,
function=function@entry=0x1d78e40 <myrocks::rdb_netbuf_to_uint32(unsigned char const*)::PRETTY_FUNCTION> "uint32 myrocks::rdb_netbuf_to_uint32(const uchar*)") at assert.c:92
#3 0x00007ffff5f4cca2 in __GI___assert_fail (assertion=0x1d6fcd5 "netbuf != nullptr", file=0x1d6fc78 "/ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/././rdb_buff.h", line=121,
function=0x1d78e40 <myrocks::rdb_netbuf_to_uint32(unsigned char const*)::PRETTY_FUNCTION> "uint32 myrocks::rdb_netbuf_to_uint32(const uchar*)") at assert.c:101
#4 0x000000000131b6ae in myrocks::rdb_netbuf_to_uint32 (netbuf=0x0) at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/././rdb_buff.h:121
#5 0x0000000001356bbf in myrocks::Rdb_key_def::unpack_record (this=0x7fffc90bb618, table=0x7fffc9064a00, buf=0x7fffc9080810 '\217' <repeats 32 times>, "\001a", '\217' <repeats 78 times>, "\230\b\b\311\377\177",
packed_key=0x7ffff7f7a040, unpack_info=0x7ffff7f7a050, verify_row_debug_checksums=@0x7fffc9021351: true) at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/rdb_datadic.cc:1514
#6 0x00000000013009b8 in myrocks::ha_rocksdb::secondary_index_read (this=0x7fffc9020d10, keyno=1, buf=0x7fffc9080810 '\217' <repeats 32 times>, "\001a", '\217' <repeats 78 times>, "\230\b\b\311\377\177")
at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/ha_rocksdb.cc:7376
#7 0x00000000013032b9 in myrocks::ha_rocksdb::index_next_with_direction (this=0x7fffc9020d10, buf=0x7fffc9080810 '\217' <repeats 32 times>, "\001a", '\217' <repeats 78 times>, "\230\b\b\311\377\177",
move_forward=true) at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/ha_rocksdb.cc:8115
#8 0x0000000001303631 in myrocks::ha_rocksdb::index_first_intern (this=0x7fffc9020d10, buf=0x7fffc9080810 '\217' <repeats 32 times>, "\001a", '\217' <repeats 78 times>, "\230\b\b\311\377\177")
at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/ha_rocksdb.cc:8223
#9 0x000000000130337b in myrocks::ha_rocksdb::index_first (this=0x7fffc9020d10, buf=0x7fffc9080810 '\217' <repeats 32 times>, "\001a", '\217' <repeats 78 times>, "\230\b\b\311\377\177")
at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/ha_rocksdb.cc:8133
#10 0x0000000001301ad1 in myrocks::ha_rocksdb::check (this=0x7fffc9020d10, thd=0x7fffd1bdf000, check_opt=0x7fffd1be3558) at /ssd/percona/myrocks/facebook-mysql-5.6/storage/rocksdb/ha_rocksdb.cc:7777
#11 0x0000000000d91ed7 in handler::ha_check (this=0x7fffc9020d10, thd=0x7fffd1bdf000, check_opt=0x7fffd1be3558) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/handler.cc:4446
#12 0x000000000117a096 in mysql_admin_table(THD *, TABLE_LIST *, HA_CHECK_OPT *, const char , thr_lock_type, bool, bool, uint, int ()(THD *, TABLE_LIST , HA_CHECK_OPT ), struct {...}, int ()(THD , TABLE_LIST ))
(thd=0x7fffd1bdf000, tables=0x7fffc90310e0, check_opt=0x7fffd1be3558, operator_name=0x1d1591c "check", lock_type=TL_READ_NO_INSERT, open_for_modify=false, repair_table_use_frm=false, extra_open_options=32,
prepare_func=0x0, operator_func=(int (handler::)(handler * const, THD , HA_CHECK_OPT )) 0xd91db8 <handler::ha_check(THD, st_ha_check_opt)>, view_operator_func=0x1022c01 <view_checksum(THD, TABLE_LIST)>)
at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_admin.cc:648
#13 0x000000000117ba37 in Sql_cmd_check_table::execute (this=0x7fffc9031650, thd=0x7fffd1bdf000) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_admin.cc:1102
#14 0x0000000000f75c2d in mysql_execute_command (thd=0x7fffd1bdf000, statement_start_time=0x7ffff7f7c208, post_parse=0x7ffff7f7c430) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_parse.cc:6143
#15 0x0000000000f79c30 in mysql_parse (thd=0x7fffd1bdf000, rawbuf=0x7fffc9031010 "check table t", length=13, parser_state=0x7ffff7f7ccd0, last_timer=0x7ffff7f7c430, async_commit=0x7ffff7f7c39b "")
at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_parse.cc:7829
#16 0x0000000000f6a2f3 in dispatch_command (command=COM_QUERY, thd=0x7fffd1bdf000, packet=0x7fffcea69001 "", packet_length=13, srv_session=0x0) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_parse.cc:1897
#17 0x0000000000f68452 in do_command (thd=0x7fffd1bdf000) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_parse.cc:1212
#18 0x0000000000f2622b in do_handle_one_connection (thd_arg=0x7fffd1bdf000) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_connect.cc:1142
#19 0x0000000000f25c42 in handle_one_connection (arg=0x7fffd1bdf000) at /ssd/percona/myrocks/facebook-mysql-5.6/sql/sql_connect.cc:963
#20 0x0000000001ad1ed7 in pfs_spawn_thread (arg=0x7ffff54ba980) at /ssd/percona/myrocks/facebook-mysql-5.6/storage/perfschema/pfs.cc:1860
#21 0x00007ffff6b0e184 in start_thread (arg=0x7ffff7f7e700) at pthread_create.c:312
#22 0x00007ffff601affd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Easily reproduced, please let me know if I can assist any further in reproducing or debugging the issue.
The text was updated successfully, but these errors were encountered: