Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bulk loaded data corrupted when restart monetdb #3282

Closed
monetdb-team opened this issue Nov 30, 2020 · 0 comments
Closed

bulk loaded data corrupted when restart monetdb #3282

monetdb-team opened this issue Nov 30, 2020 · 0 comments

Comments

@monetdb-team
Copy link

@monetdb-team monetdb-team commented Nov 30, 2020

Date: 2013-05-14 10:02:34 +0200
From: Inho Kim <<georgios.kim>>
To: SQL devs <>
Version: 11.15.7 (Feb2013-SP2)
CC: @drstmane

Last updated: 2013-07-03 08:48:01 +0200

Comment 18720

Date: 2013-05-14 10:02:34 +0200
From: Inho Kim <<georgios.kim>>

User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Build Identifier:

Alter load 65k rows with bulk load (copy), query will succeed and data was correct. When restart mserver5, same query returns below warning messages.

"1 field truncated!\n note: to disable dropping columns and/or truncating fields use \w-1\n write error"

Reproducible: Always

Steps to Reproduce:

[inho@ChakraMonet tmp]$ mclient -u monetdb -d test
password:
Welcome to mclient, the MonetDB/SQL interactive terminal (Feb2013-SP2)
Database: MonetDB v11.15.7 (Feb2013-SP2), 'mapi:monetdb://ChakraMonet:50002/test'
Type \q to quit, ? for a list of available commands
auto commit mode: on
sql></tmp/test.sql
operation successful
655360 affected rows
sql>select * from test limit 20 offset 655350;
+------+--------+-------+
| id | name | seq |
+======+========+=======+
| id0 | Test 0 | 1 |
| id2 | Test 2 | 2 |
| id3 | Test 3 | 3 |
| id4 | Test 4 | 4 |
| id5 | Test 5 | 5 |
| id6 | Test 6 | 6 |
| id7 | Test 7 | 7 |
| id9 | Test 9 | 9 |
| id8 | Test 8 | 99999 |
| id9 | Test 9 | 9 |
+------+--------+-------+
10 tuples (50.898ms)
sql>\q
[inho@ChakraMonet tmp]$ monetdb stop test
stopping database 'test'... done
[inho@ChakraMonet tmp]$ monetdb start test
starting database 'test'... done
[inho@ChakraMonet tmp]$ !mclient
mclient -u monetdb -d test
password:
Welcome to mclient, the MonetDB/SQL interactive terminal (Feb2013-SP2)
Database: MonetDB v11.15.7 (Feb2013-SP2), 'mapi:monetdb://ChakraMonet:50002/test'
Type \q to quit, ? for a list of available commands
auto commit mode: on
sql>select * from test limit 20 offset 655350;
+------+--------+-----------+
| id | name | seq |
+======+========+===========+
| id0 | Test 0 | 1 |
| id2 | Test 2 | 2 |
| id3 | Test 3 | 3 |
| id4 | Test 4 | 4 |
| id5 | Test 5 | 5 |
| id6 | Test 6 | 6 |
| id7 | Test 7 | 7 |
| id9 | Test 9 | 9 |
| id8 | Test 8 | 99999 |
| 녋랁u |10 tuples (42.354ms) !1 field truncated!
note: to disable dropping columns and/or truncating fields use \w-1
write error
sql>

Comment 18721

Date: 2013-05-14 10:04:22 +0200
From: Inho Kim <<georgios.kim>>

Created attachment 194
bulk load data

Attached file: test.sql.gz (application/x-gzip, 74157 bytes)
Description: bulk load data

Comment 18722

Date: 2013-05-14 10:15:31 +0200
From: @drstmane

Is your original output indeed correct, in particular the double "| id9 | Test 9 | 9 |" record (the second occurrence of which seem to be corrupted after the restart)?

Can you share the content of your monetdbd / merovingian log file that is related to stopping and restarting the server after the bulk load?

Comment 18723

Date: 2013-05-14 10:27:25 +0200
From: Inho Kim <<georgios.kim>>

Is your original output indeed correct, in particular the double "| id9 | Test 9 >| 9 |" record (the second occurrence of which seem to be corrupted after the >restart)?

yes..

Can you share the content of your monetdbd / merovingian log file that is related >to stopping and restarting the server after the bulk load?

I already uploaded test.sql.gz file ( for bulk load )

Comment 18724

Date: 2013-05-14 10:28:21 +0200
From: Inho Kim <<georgios.kim>>

Created attachment 195
merovingian.log

Attached file: merovingian.log (text/plain, 5290 bytes)
Description: merovingian.log

Comment 18725

Date: 2013-05-14 12:50:01 +0200
From: @sjoerdmullender

After loading the data and restarting the server, I get an assertion error:
mserver5: monetdb5/modules/atoms/str.c:1357: strLength: Assertion `l <2147483647' failed.
I'll investigate a bit more.

Comment 18726

Date: 2013-05-14 15:11:48 +0200
From: MonetDB Mercurial Repository <>

Changeset 83ada82a21d8 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=83ada82a21d8

Changeset description:

Off-by-one error: don't extend if file already correct size.
This fixes bug #3282.
Extending a file is done by seeking to the desired size and writing a
byte.  In the case encountered here, the file was already the correct
size, but a byte was written anyway, overwriting the last byte in the
file and thereby corrupting the data.

Comment 18727

Date: 2013-05-14 15:49:09 +0200
From: MonetDB Mercurial Repository <>

Changeset 934a44f61041 made by Sjoerd Mullender sjoerd@acm.org in the MonetDB repo, refers to this bug.

For complete details, see http//devmonetdborg/hg/MonetDB?cmd=changeset;node=934a44f61041

Changeset description:

Test for bug #3282.

Comment 18728

Date: 2013-05-14 15:57:35 +0200
From: @sjoerdmullender

Bug is fixed, test is added.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant