Fix leadership lost just before the WalFrames replication hook #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When the leader loses leadership just before running the WalFrames replication hook (typically to commit a transaction), it now properly marks the transaction as stale (so the automatic WAL rollback initiated by SQLite will effectively be a no-op) and start a surrogate follower (so the next leader can roll back the leftover transaction that it finds). Also, the demo program has been changed to handle tx.Commit() failures differently: in that case there's no need to call tx.Rollback(), as that is automatically handled by SQLite. Fixes #2.
freeekanayaka
deleted the
handle-leadership-lost-just-before-wal-frames-hook
branch
mjeanson
added a commit
to mjeanson/dqlite
that referenced
this pull request
Aug 3, 2020
On an up to date Ubuntu 20.04 with raft 0.9.24, libco v20 and sqlite
3.32.3+replication4 the tests fail with errors like:
==96452==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x607000000fd8 in thread T0
#0 0x7f82cfea37cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
canonical#1 0x564796ec2065 in mem_fault_free test/lib/heap.c:57
canonical#2 0x7f82cfc8370e in sqlite3_free sqlite3.c:28007
canonical#3 0x7f82cfc8370e in sqlite3_free sqlite3.c:27999
canonical#4 0x564796dcd89c in applyCb src/replication.c:116
canonical#5 0x7f82cfc483ad in convertFailApply src/convert.c:59
canonical#6 0x7f82cfc483ad in convertClearLeader src/convert.c:95
canonical#7 0x7f82cfc483ad in convertClear src/convert.c:124
canonical#8 0x7f82cfc4858e in convertToUnavailable src/convert.c:218
canonical#9 0x7f82cfc4a8a4 in raft_close src/raft.c:97
canonical#10 0x564796dec401 in exec__tear_down test/unit/test_conn.c:302
canonical#11 0x564796ec5247 in munit_test_runner_exec test/lib/munit.c:1203
canonical#12 0x564796ec6b2c in munit_test_runner_run_test_with_params test/lib/munit.c:1357
canonical#13 0x564796ec7c3c in munit_test_runner_run_test test/lib/munit.c:1585
canonical#14 0x564796ec8505 in munit_test_runner_run_suite test/lib/munit.c:1678
canonical#15 0x564796ec85a5 in munit_test_runner_run_suite test/lib/munit.c:1687
canonical#16 0x564796ec85a5 in munit_test_runner_run_suite test/lib/munit.c:1687
canonical#17 0x564796ec9c43 in munit_test_runner_run test/lib/munit.c:1697
canonical#18 0x564796ec9c43 in munit_suite_main_custom test/lib/munit.c:2027
canonical#19 0x564796ecaf77 in munit_suite_main test/lib/munit.c:2055
canonical#20 0x564796db2d92 in main test/unit/main.c:3
canonical#21 0x7f82cfa060b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
canonical#22 0x564796db30dd in _start (/home/mjeanson/Git/lxc/dqlite/unit-test.test+0xbd0dd)
0x607000000fd8 is located 8 bytes to the left of 72-byte region [0x607000000fe0,0x607000001028)
allocated by thread T0 here:
#0 0x7f82cfea3bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
SUMMARY: AddressSanitizer: bad-free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf) in __interceptor_free
Signed-off-by: Michael Jeanson <mjeanson@debian.org>
mjeanson
added a commit
to mjeanson/dqlite
that referenced
this pull request
Aug 3, 2020
On an up to date Ubuntu 20.04 with raft 0.9.24, libco v20 and sqlite
3.32.3+replication4 the tests fail with errors like:
==96452==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x607000000fd8 in thread T0
#0 0x7f82cfea37cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
canonical#1 0x564796ec2065 in mem_fault_free test/lib/heap.c:57
canonical#2 0x7f82cfc8370e in sqlite3_free sqlite3.c:28007
canonical#3 0x7f82cfc8370e in sqlite3_free sqlite3.c:27999
canonical#4 0x564796dcd89c in applyCb src/replication.c:116
canonical#5 0x7f82cfc483ad in convertFailApply src/convert.c:59
canonical#6 0x7f82cfc483ad in convertClearLeader src/convert.c:95
canonical#7 0x7f82cfc483ad in convertClear src/convert.c:124
canonical#8 0x7f82cfc4858e in convertToUnavailable src/convert.c:218
canonical#9 0x7f82cfc4a8a4 in raft_close src/raft.c:97
canonical#10 0x564796dec401 in exec__tear_down test/unit/test_conn.c:302
canonical#11 0x564796ec5247 in munit_test_runner_exec test/lib/munit.c:1203
canonical#12 0x564796ec6b2c in munit_test_runner_run_test_with_params test/lib/munit.c:1357
canonical#13 0x564796ec7c3c in munit_test_runner_run_test test/lib/munit.c:1585
canonical#14 0x564796ec8505 in munit_test_runner_run_suite test/lib/munit.c:1678
canonical#15 0x564796ec85a5 in munit_test_runner_run_suite test/lib/munit.c:1687
canonical#16 0x564796ec85a5 in munit_test_runner_run_suite test/lib/munit.c:1687
canonical#17 0x564796ec9c43 in munit_test_runner_run test/lib/munit.c:1697
canonical#18 0x564796ec9c43 in munit_suite_main_custom test/lib/munit.c:2027
canonical#19 0x564796ecaf77 in munit_suite_main test/lib/munit.c:2055
canonical#20 0x564796db2d92 in main test/unit/main.c:3
canonical#21 0x7f82cfa060b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
canonical#22 0x564796db30dd in _start (/home/mjeanson/Git/lxc/dqlite/unit-test.test+0xbd0dd)
0x607000000fd8 is located 8 bytes to the left of 72-byte region [0x607000000fe0,0x607000001028)
allocated by thread T0 here:
#0 0x7f82cfea3bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
SUMMARY: AddressSanitizer: bad-free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf) in __interceptor_free
Signed-off-by: Michael Jeanson <mjeanson@debian.org>
freeekanayaka
pushed a commit
that referenced
this pull request
Aug 3, 2020
On an up to date Ubuntu 20.04 with raft 0.9.24, libco v20 and sqlite
3.32.3+replication4 the tests fail with errors like:
==96452==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x607000000fd8 in thread T0
#0 0x7f82cfea37cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
#1 0x564796ec2065 in mem_fault_free test/lib/heap.c:57
#2 0x7f82cfc8370e in sqlite3_free sqlite3.c:28007
#3 0x7f82cfc8370e in sqlite3_free sqlite3.c:27999
#4 0x564796dcd89c in applyCb src/replication.c:116
#5 0x7f82cfc483ad in convertFailApply src/convert.c:59
#6 0x7f82cfc483ad in convertClearLeader src/convert.c:95
#7 0x7f82cfc483ad in convertClear src/convert.c:124
#8 0x7f82cfc4858e in convertToUnavailable src/convert.c:218
#9 0x7f82cfc4a8a4 in raft_close src/raft.c:97
#10 0x564796dec401 in exec__tear_down test/unit/test_conn.c:302
#11 0x564796ec5247 in munit_test_runner_exec test/lib/munit.c:1203
#12 0x564796ec6b2c in munit_test_runner_run_test_with_params test/lib/munit.c:1357
#13 0x564796ec7c3c in munit_test_runner_run_test test/lib/munit.c:1585
#14 0x564796ec8505 in munit_test_runner_run_suite test/lib/munit.c:1678
#15 0x564796ec85a5 in munit_test_runner_run_suite test/lib/munit.c:1687
#16 0x564796ec85a5 in munit_test_runner_run_suite test/lib/munit.c:1687
#17 0x564796ec9c43 in munit_test_runner_run test/lib/munit.c:1697
#18 0x564796ec9c43 in munit_suite_main_custom test/lib/munit.c:2027
#19 0x564796ecaf77 in munit_suite_main test/lib/munit.c:2055
#20 0x564796db2d92 in main test/unit/main.c:3
#21 0x7f82cfa060b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#22 0x564796db30dd in _start (/home/mjeanson/Git/lxc/dqlite/unit-test.test+0xbd0dd)
0x607000000fd8 is located 8 bytes to the left of 72-byte region [0x607000000fe0,0x607000001028)
allocated by thread T0 here:
#0 0x7f82cfea3bc8 in malloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10dbc8)
SUMMARY: AddressSanitizer: bad-free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf) in __interceptor_free
Signed-off-by: Michael Jeanson <mjeanson@debian.org>
mjeanson
added a commit
to mjeanson/dqlite
that referenced
this pull request
Aug 3, 2020
==199893==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x619000004178 in thread T0
#0 0x7fd51bdab7cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
canonical#1 0x56243f445145 in mem_fault_free test/lib/heap.c:57
canonical#2 0x7fd51bb8b70e in sqlite3_free sqlite3.c:28007
canonical#3 0x7fd51bb8b70e in sqlite3_free sqlite3.c:27999
canonical#4 0x56243f43a95e in test_vfs_pollAfterWriteTransaction test/integration/test_vfs.c:238
canonical#5 0x56243f44a1a9 in munit_test_runner_exec test/lib/munit.c:1195
canonical#6 0x56243f44b198 in munit_test_runner_run_test_with_params test/lib/munit.c:1357
canonical#7 0x56243f44d81c in munit_test_runner_run_test test/lib/munit.c:1585
canonical#8 0x56243f44e917 in munit_test_runner_run_suite test/lib/munit.c:1678
canonical#9 0x56243f44eaf1 in munit_test_runner_run_suite test/lib/munit.c:1687
canonical#10 0x56243f44ecb8 in munit_test_runner_run test/lib/munit.c:1697
canonical#11 0x56243f45253d in munit_suite_main_custom test/lib/munit.c:2027
canonical#12 0x56243f452bac in munit_suite_main test/lib/munit.c:2055
canonical#13 0x56243f444f0d in main test/integration/main.c:3
canonical#14 0x7fd51b88b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
canonical#15 0x56243f4322fd in _start (/home/mjeanson/Git/lxc/dqlite/integration-test+0x142fd)
0x619000004178 is located 8 bytes to the left of 1024-byte region [0x619000004180,0x619000004580)
allocated by thread T0 here:
#0 0x7fd51bdabdc6 in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10ddc6)
canonical#1 0x56243f4471d6 in munit_malloc_ex test/lib/munit.c:280
canonical#2 0x56243f43a39f in test_vfs_pollAfterWriteTransaction test/integration/test_vfs.c:230
canonical#3 0x56243f44a1a9 in munit_test_runner_exec test/lib/munit.c:1195
canonical#4 0x56243f44b198 in munit_test_runner_run_test_with_params test/lib/munit.c:1357
canonical#5 0x56243f44d81c in munit_test_runner_run_test test/lib/munit.c:1585
canonical#6 0x56243f44e917 in munit_test_runner_run_suite test/lib/munit.c:1678
canonical#7 0x56243f44eaf1 in munit_test_runner_run_suite test/lib/munit.c:1687
canonical#8 0x56243f44ecb8 in munit_test_runner_run test/lib/munit.c:1697
canonical#9 0x56243f45253d in munit_suite_main_custom test/lib/munit.c:2027
canonical#10 0x56243f452bac in munit_suite_main test/lib/munit.c:2055
canonical#11 0x56243f444f0d in main test/integration/main.c:3
canonical#12 0x7fd51b88b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
SUMMARY: AddressSanitizer: bad-free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf) in __interceptor_free
Signed-off-by: Michael Jeanson <mjeanson@debian.org>
freeekanayaka
pushed a commit
that referenced
this pull request
Aug 4, 2020
==199893==ERROR: AddressSanitizer: attempting free on address which was not malloc()-ed: 0x619000004178 in thread T0
#0 0x7fd51bdab7cf in __interceptor_free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf)
#1 0x56243f445145 in mem_fault_free test/lib/heap.c:57
#2 0x7fd51bb8b70e in sqlite3_free sqlite3.c:28007
#3 0x7fd51bb8b70e in sqlite3_free sqlite3.c:27999
#4 0x56243f43a95e in test_vfs_pollAfterWriteTransaction test/integration/test_vfs.c:238
#5 0x56243f44a1a9 in munit_test_runner_exec test/lib/munit.c:1195
#6 0x56243f44b198 in munit_test_runner_run_test_with_params test/lib/munit.c:1357
#7 0x56243f44d81c in munit_test_runner_run_test test/lib/munit.c:1585
#8 0x56243f44e917 in munit_test_runner_run_suite test/lib/munit.c:1678
#9 0x56243f44eaf1 in munit_test_runner_run_suite test/lib/munit.c:1687
#10 0x56243f44ecb8 in munit_test_runner_run test/lib/munit.c:1697
#11 0x56243f45253d in munit_suite_main_custom test/lib/munit.c:2027
#12 0x56243f452bac in munit_suite_main test/lib/munit.c:2055
#13 0x56243f444f0d in main test/integration/main.c:3
#14 0x7fd51b88b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#15 0x56243f4322fd in _start (/home/mjeanson/Git/lxc/dqlite/integration-test+0x142fd)
0x619000004178 is located 8 bytes to the left of 1024-byte region [0x619000004180,0x619000004580)
allocated by thread T0 here:
#0 0x7fd51bdabdc6 in calloc (/lib/x86_64-linux-gnu/libasan.so.5+0x10ddc6)
#1 0x56243f4471d6 in munit_malloc_ex test/lib/munit.c:280
#2 0x56243f43a39f in test_vfs_pollAfterWriteTransaction test/integration/test_vfs.c:230
#3 0x56243f44a1a9 in munit_test_runner_exec test/lib/munit.c:1195
#4 0x56243f44b198 in munit_test_runner_run_test_with_params test/lib/munit.c:1357
#5 0x56243f44d81c in munit_test_runner_run_test test/lib/munit.c:1585
#6 0x56243f44e917 in munit_test_runner_run_suite test/lib/munit.c:1678
#7 0x56243f44eaf1 in munit_test_runner_run_suite test/lib/munit.c:1687
#8 0x56243f44ecb8 in munit_test_runner_run test/lib/munit.c:1697
#9 0x56243f45253d in munit_suite_main_custom test/lib/munit.c:2027
#10 0x56243f452bac in munit_suite_main test/lib/munit.c:2055
#11 0x56243f444f0d in main test/integration/main.c:3
#12 0x7fd51b88b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
SUMMARY: AddressSanitizer: bad-free (/lib/x86_64-linux-gnu/libasan.so.5+0x10d7cf) in __interceptor_free
Signed-off-by: Michael Jeanson <mjeanson@debian.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When the leader loses leadership just before running the WalFrames
replication hook (typically to commit a transaction), it now properly
marks the transaction as stale (so the automatic WAL rollback
initiated by SQLite will effectively be a no-op) and start a surrogate
follower (so the next leader can roll back the leftover transaction
that it finds).
Also, the demo program has been changed to handle tx.Commit() failures
differently: in that case there's no need to call tx.Rollback(), as
that is automatically handled by SQLite.
Fixes #2 and #4.