Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RQG tests failed with "Got error 122 from storage engine" error #238

Closed
mariadb-DanielLee opened this issue Apr 28, 2016 · 6 comments
Closed

RQG tests failed with "Got error 122 from storage engine" error #238

mariadb-DanielLee opened this issue Apr 28, 2016 · 6 comments
Assignees
@jkedgar
Labels
bug high-pri myrocks

Comments

@mariadb-DanielLee
Copy link

Build tested:

myrocks@rocksdb:~/builds/reptest/mysql-5.6$ git show
commit e4f22e0
Author: Jay Edgar jkedgar@fb.com
Date: Wed Apr 6 12:48:20 2016 -0700

The following statements returned a storage error. In turn, RQG reports a database corruption error:
#2016-04-28T14:28:38 [44278] Test completed with failure status STATUS_DATABASE_CORRUPTION (105)

mysql> SELECT col_enum_utf8 FROM table1_int_autoinc WHERE 1 = 1;
ERROR 1030 (HY000): Got error 122 from storage engine

mysql> INSERT INTO table100_key_pk_parts_2_int_autoinc ( col_int_key, col_int ) SELECT col_int, col_int_key FROM table100_int_autoinc WHERE pk > 10 LIMIT 3;
ERROR 1030 (HY000): Got error 122 from storage engine

mysql> desc table1_int_autoinc;
+-------------------------------------+---------------------------------------------------------------------------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------------------------+---------------------------------------------------------------------------------------------------------------+------+-----+---------+----------------+
| col_enum_latin1_key | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_datetime_key_default_null | datetime | YES | MUL | NULL | |
| col_int_default_null | int(11) | YES | | NULL | |
| col_enum_utf8_key_default_null | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_int_unsigned_key | int(10) unsigned | YES | MUL | NULL | |
| col_int_unsigned_key_default_null | int(10) unsigned | YES | MUL | NULL | |
| col_set_utf8 | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_enum_latin1_default_null | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_int | int(11) | YES | | NULL | |
| col_char_64_latin1_not_null | char(64) | NO | | NULL | |
| col_set_latin1_not_null | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | | NULL | |
| col_char_64_utf8_key_default_null | char(64) | YES | MUL | NULL | |
| col_set_latin1_key | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_enum_latin1 | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_enum_utf8_default_null | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_set_latin1_default_null | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_char_64_utf8_default_null | char(64) | YES | | NULL | |
| col_datetime_key | datetime | YES | MUL | NULL | |
| col_set_utf8_default_null | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_set_utf8_key | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_char_64_latin1 | char(64) | YES | | NULL | |
| col_char_64_utf8_not_null | char(64) | NO | | NULL | |
| col_int_not_null_key | int(11) | NO | MUL | NULL | |
| col_int_unsigned_not_null_key | int(10) unsigned | NO | MUL | NULL | |
| col_char_64_latin1_key_default_null | char(64) | YES | MUL | NULL | |
| col_int_unsigned | int(10) unsigned | YES | | NULL | |
| col_enum_utf8_key | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_enum_latin1_key_default_null | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_set_utf8_not_null_key | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | MUL | NULL | |
| col_char_64_utf8 | char(64) | YES | | NULL | |
| col_set_latin1_key_default_null | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_datetime_not_null | datetime | NO | | NULL | |
| col_int_unsigned_not_null | int(10) unsigned | NO | | NULL | |
| col_enum_utf8_not_null_key | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | MUL | NULL | |
| col_char_64_latin1_default_null | char(64) | YES | | NULL | |
| col_enum_latin1_not_null | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | | NULL | |
| col_int_not_null | int(11) | NO | | NULL | |
| col_set_latin1 | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_set_utf8_not_null | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | | NULL | |
| col_datetime | datetime | YES | | NULL | |
| col_datetime_default_null | datetime | YES | | NULL | |
| col_int_key | int(11) | YES | MUL | NULL | |
| col_set_utf8_key_default_null | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | MUL | NULL | |
| col_char_64_latin1_key | char(64) | YES | MUL | NULL | |
| col_char_64_utf8_key | char(64) | YES | MUL | NULL | |
| col_enum_latin1_not_null_key | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | MUL | NULL | |
| pk | int(11) | NO | PRI | NULL | auto_increment |
| col_set_latin1_not_null_key | set('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | MUL | NULL | |
| col_int_unsigned_default_null | int(10) unsigned | YES | | NULL | |
| col_enum_utf8_not_null | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | NO | | NULL | |
| col_char_64_utf8_not_null_key | char(64) | NO | MUL | NULL | |
| col_datetime_not_null_key | datetime | NO | MUL | NULL | |
| col_char_64_latin1_not_null_key | char(64) | NO | MUL | NULL | |
| col_enum_utf8 | enum('a','b','c','d','e','f','g','h','i','j','k','l','m','n','o','p','q','r','s','t','u','v','w','x','y','z') | YES | | NULL | |
| col_int_key_default_null | int(11) | YES | MUL | NULL | |
+-------------------------------------+---------------------------------------------------------------------------------------------------------------+------+-----+---------+----------------+
55 rows in set (0.00 sec)
@yoshinorim
Copy link
Contributor

@dleeyh : Could you paste the exact rqg command you reproduced?

@mariadb-DanielLee
Copy link
Author 

  perl gentest.pl --dsn=dbi:mysql:host=localhost:port=3306:user=root:database=$testDB:mysql_socket=3307 --gendata=conf/transactions/combinations.zz --grammar=conf/transactions/repeatable_read.yy --threads=3 --queries=10000 --sqltrace

  perl gentest.pl --dsn=dbi:mysql:host=localhost:port=3306:user=root:database=$testDB:mysql_socket=3307 --gendata=conf/transactions/combinations.zz --grammar=conf/transactions/transactions-flat.yy --threads=3 --queries=10000 --sqltrace

@alxyang alxyang self-assigned this May 4, 2016
@alxyang
Copy link
Contributor

alxyang commented May 4, 2016

@dleeyh can you check if #239 resolved this issue

@yoshinorim
Copy link
Contributor

I debugged and it failed at myrocks::ha_rocksdb::convert_record_from_storage_format().

    /* Skip the bytes we need to skip */
    if (it->skip && !reader.read(it->skip))
      return HA_ERR_INTERNAL_ERROR;

it->skip was 282, and reader->m_len was 281. reader.read() returned nullptr because m_len was smaller than the argument, so the function returned HA_ERR_INTERNAL_ERROR.

@alxyang alxyang assigned jkedgar and unassigned alxyang May 4, 2016
@jkedgar
Copy link
Contributor

jkedgar commented May 4, 2016

I think I've found the problem. The field_decoders code, which allows us to skip over fields that we don't need to decode, doesn't handle NULL fields in the skipped data correctly. I can reproduce the 'Got error 122 from storage engine' with this simple script.

create table t1 (a int, b int);
insert into t1 values(NULL, 1);
select b from t1;

I'm very surprised none of our existing tests select specific columns where earlier columns are NULL. I'll work on a fix for this.

@jkedgar
Copy link
Contributor

jkedgar commented May 4, 2016

https://reviews.facebook.net/D57633

@jkedgar jkedgar mentioned this issue May 4, 2016
Closed
jkedgar added a commit that referenced this issue May 4, 2016
@jkedgar
Fix a corruption error related to NULL columns.
1ef62b8 
Summary:
The MyRocks code has some field_decoders structures that allow columns that are not needed to not be decoded when converting from the on-disk data format to the MySQL in-memory format.  These were not set up correctly when nullable columns were part of the row and would cause us to attempt to skip too many bytes in the row - causing either incorrect data or an error indicating corruption.  (Issue #238)

Squash with: https://reviews.facebook.net/D56115

Test Plan: Updated rocksdb.col_opt_null to add a query that fails with the old code and verified that it succeeded with the new code.

Reviewers: hermanlee4, yoshinorim, spetrunia

Reviewed By: spetrunia

Subscribers: webscalesql-eng

Differential Revision: https://reviews.facebook.net/D57633
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jkedgar jkedgar

Labels
bug high-pri myrocks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@yoshinorim @alxyang @jkedgar @mariadb-DanielLee @vasilep