Repeatable {in, up}sert with memtx transaction manager enabled causes transaction conflict #7217

CuriousGeorgiy · 2022-05-30T12:38:23Z

Steps to reproduce

#!/usr/bin/env tarantool

os.execute('rm -rf *.snap *.xlog *.vylog 512')

local log = require('log')
local txn_proxy = require('txn_proxy')
local yaml = require('yaml')

box.cfg{memtx_use_mvcc_engine = true}

box.schema.space.create('s')
box.space.s:create_index('pk')

local tx1 = txn_proxy:new()
tx1('box.begin()')

local tx2 = txn_proxy:new()
tx2('box.begin()')

tx1('box.space.s:replace{0, 0}')

tx2('box.space.s:replace{0, 1}')
tx2('box.space.s:delete{0}')
tx2('box.space.s:insert{0, 2}')

tx1('box.commit()')
log.info("%s", yaml.encode(tx2('box.commit()')))

os.exit()

#!/usr/bin/env tarantool

os.execute('rm -rf *.snap *.xlog *.vylog 512')

local log = require('log')
local txn_proxy = require('txn_proxy')
local yaml = require('yaml')

box.cfg{memtx_use_mvcc_engine = true}

box.schema.space.create('s')
box.space.s:create_index('pk')

local tx1 = txn_proxy:new()
tx1('box.begin()')

local tx2 = txn_proxy:new()
tx2('box.begin()')

tx1('box.space.s:replace{0, 0}')

tx2('box.space.s:replace{0, 1}')
tx2('box.space.s:delete{0}')
tx2('box.space.s:upsert({0, 2}, {{"=", 2, 2}})')

tx1('box.commit()')
log.info("%s", yaml.encode(tx2('box.commit()')))

os.exit()

Actual behavior

 ---
- {'error': 'Transaction has been aborted by conflict'}
...

Expected behavior

 ---
...

Current implementation of tracking statements that delete a story has a flaw, consider the following example: tx1('box.space.s:replace{0, 0}') -- statement 1 tx2('box.space.s:replace{0, 1}') -- statement 2 tx2('box.space.s:delete{0}') -- statement 3 tx2('box.space.s:replace{0, 2}') -- statement 4 When statement 1 is prepared, both statements 2 and 4 will be linked to the delete statement list of {0, 0}'s story, though, apparently, statement 4 does not delete {0, 0}. Let us notice the following: statement 4 is "pure" in the sense that, in the transaction's scope, it is guaranteed not to replace any tuple — we can retrieve this information when we check where the insert statement violates replacement rules, use it to determine "pure" insert statements, and skip them later on when, during preparation of insert statements, we handle other insert statements which assume they do not replace anything (i.e., have no visible old tuple). We also need to fix relinking of delete statements from the older story (in terms of the history chain) to the new one during preparation of insert statements: a statement needs to be relinked iff it comes from a different transaction (to be precise, there must, actually, be no more than one delete statement from the same transaction). Additionally, added assertions to verify the invariant that the story's add (delete) psn is equal to the psn of the add (delete) statement's transaction psn and to verify that no self-conflicting of transactions occurs. Closes tarantool#7214 Closes tarantool#7217 NO_DOC=bugfix

When full scans are checked on writes, we must only conflict transactions other than the one that did the full scan. NO_CHANGELOG=internal bugfix NO_DOC=bugfix Closes tarantool#7217 Closes tarantool#7221

Current implementation of tracking statements that delete a story has a flaw, consider the following example: tx1('box.space.s:replace{0, 0}') -- statement 1 tx2('box.space.s:replace{0, 1}') -- statement 2 tx2('box.space.s:delete{0}') -- statement 3 tx2('box.space.s:replace{0, 2}') -- statement 4 When statement 1 is prepared, both statements 2 and 4 will be linked to the delete statement list of {0, 0}'s story, though, apparently, statement 4 does not delete {0, 0}. Let us notice the following: statement 4 is "pure" in the sense that, in the transaction's scope, it is guaranteed not to replace any tuple — we can retrieve this information when we check where the insert statement violates replacement rules, use it to determine "pure" insert statements, and skip them later on when, during preparation of insert statements, we handle other insert statements which assume they do not replace anything (i.e., have no visible old tuple). We also need to fix relinking of delete statements from the older story (in terms of the history chain) to the new one during preparation of insert statements: a statement needs to be relinked iff it comes from a different transaction (to be precise, there must, actually, be no more than one delete statement from the same transaction). Additionally, added assertions to verify the invariant that the story's add (delete) psn is equal to the psn of the add (delete) statement's transaction psn. Closes tarantool#7214 Closes tarantool#7217 NO_DOC=bugfix

Current implementation of tracking statements that delete a story has a flaw, consider the following example: tx1('box.space.s:replace{0, 0}') -- statement 1 tx2('box.space.s:replace{0, 1}') -- statement 2 tx2('box.space.s:delete{0}') -- statement 3 tx2('box.space.s:replace{0, 2}') -- statement 4 When statement 1 is prepared, both statements 2 and 4 will be linked to the delete statement list of {0, 0}'s story, though, apparently, statement 4 does not delete {0, 0}. Let us notice the following: statement 4 is "pure" in the sense that, in the transaction's scope, it is guaranteed not to replace any tuple — we can retrieve this information when we check where the insert statement violates replacement rules, use it to determine "pure" insert statements, and skip them later on when, during preparation of insert statements, we handle other insert statements which assume they do not replace anything (i.e., have no visible old tuple). On the contrary, statements 1 and 2 are "dirty": they assume that they replaced nothing (i.e., there was no visible tuple in the index) — when one of them gets prepared — the other one needs to be either aborted or relinked to replace the prepared tuple. We also need to fix relinking of delete statements from the older story (in terms of the history chain) to the new one during preparation of insert statements: a statement needs to be relinked iff it comes from a different transaction (to be precise, there must, actually, be no more than one delete statement from the same transaction). Additionally, add assertions to verify the invariant that the story's add (delete) psn is equal to the psn of the add (delete) statement's transaction psn. Closes tarantool#7214 Closes tarantool#7217 NO_DOC=bugfix

Current implementation of tracking statements that delete a story has a flaw, consider the following example: tx1('box.space.s:replace{0, 0}') -- statement 1 tx2('box.space.s:replace{0, 1}') -- statement 2 tx2('box.space.s:delete{0}') -- statement 3 tx2('box.space.s:replace{0, 2}') -- statement 4 When statement 1 is prepared, both statements 2 and 4 will be linked to the delete statement list of {0, 0}'s story, though, apparently, statement 4 does not delete {0, 0}. Let us notice the following: statement 4 is "pure" in the sense that, in the transaction's scope, it is guaranteed not to replace any tuple — we can retrieve this information when we check where the insert statement violates replacement rules, use it to determine "pure" insert statements, and skip them later on when, during preparation of insert statements, we handle other insert statements which assume they do not replace anything (i.e., have no visible old tuple). On the contrary, statements 1 and 2 are "dirty": they assume that they replaced nothing (i.e., there was no visible tuple in the index) — when one of them gets prepared — the other one needs to be either aborted or relinked to replace the prepared tuple. We also need to fix relinking of delete statements from the older story (in terms of the history chain) to the new one during preparation of insert statements: a statement needs to be relinked iff it comes from a different transaction (to be precise, there must, actually, be no more than one delete statement from the same transaction). Additionally, add assertions to verify the invariant that the story's add (delete) psn is equal to the psn of the add (delete) statement's transaction psn. Closes #7214 Closes #7217 NO_DOC=bugfix

Current implementation of tracking statements that delete a story has a flaw, consider the following example: tx1('box.space.s:replace{0, 0}') -- statement 1 tx2('box.space.s:replace{0, 1}') -- statement 2 tx2('box.space.s:delete{0}') -- statement 3 tx2('box.space.s:replace{0, 2}') -- statement 4 When statement 1 is prepared, both statements 2 and 4 will be linked to the delete statement list of {0, 0}'s story, though, apparently, statement 4 does not delete {0, 0}. Let us notice the following: statement 4 is "pure" in the sense that, in the transaction's scope, it is guaranteed not to replace any tuple — we can retrieve this information when we check where the insert statement violates replacement rules, use it to determine "pure" insert statements, and skip them later on when, during preparation of insert statements, we handle other insert statements which assume they do not replace anything (i.e., have no visible old tuple). On the contrary, statements 1 and 2 are "dirty": they assume that they replaced nothing (i.e., there was no visible tuple in the index) — when one of them gets prepared — the other one needs to be either aborted or relinked to replace the prepared tuple. We also need to fix relinking of delete statements from the older story (in terms of the history chain) to the new one during preparation of insert statements: a statement needs to be relinked iff it comes from a different transaction (to be precise, there must, actually, be no more than one delete statement from the same transaction). Additionally, add assertions to verify the invariant that the story's add (delete) psn is equal to the psn of the add (delete) statement's transaction psn. Closes #7214 Closes #7217 NO_DOC=bugfix (cherry picked from commit 654cf49)

Current implementation of tracking statements that delete a story has a flaw, consider the following example: tx1('box.space.s:replace{0, 0}') -- statement 1 tx2('box.space.s:replace{0, 1}') -- statement 2 tx2('box.space.s:delete{0}') -- statement 3 tx2('box.space.s:replace{0, 2}') -- statement 4 When statement 1 is prepared, both statements 2 and 4 will be linked to the delete statement list of {0, 0}'s story, though, apparently, statement 4 does not delete {0, 0}. Let us notice the following: statement 4 is "pure" in the sense that, in the transaction's scope, it is guaranteed not to replace any tuple — we can retrieve this information when we check where the insert statement violates replacement rules, use it to determine "pure" insert statements, and skip them later on when, during preparation of insert statements, we handle other insert statements which assume they do not replace anything (i.e., have no visible old tuple). On the contrary, statements 1 and 2 are "dirty": they assume that they replaced nothing (i.e., there was no visible tuple in the index) — when one of them gets prepared — the other one needs to be either aborted or relinked to replace the prepared tuple. We also need to fix relinking of delete statements from the older story (in terms of the history chain) to the new one during preparation of insert statements: a statement needs to be relinked iff it comes from a different transaction (to be precise, there must, actually, be no more than one delete statement from the same transaction). Additionally, add assertions to verify the invariant that the story's add (delete) psn is equal to the psn of the add (delete) statement's transaction psn. Closes tarantool#7214 Closes tarantool#7217 NO_DOC=bugfix

CuriousGeorgiy added bug Something isn't working complicated memtx teamC mvcc labels May 30, 2022

CuriousGeorgiy added this to the 2.10.1 milestone May 30, 2022

CuriousGeorgiy self-assigned this May 30, 2022

CuriousGeorgiy mentioned this issue May 31, 2022

memtx: fix story delete statement list #7215

Merged

CuriousGeorgiy mentioned this issue Jun 6, 2022

memtx: fix self-conflicting of transactions #7236

Merged

alyapunov closed this as completed in #7215 Jul 6, 2022

alyapunov added the 3sp label Jul 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repeatable {in, up}sert with memtx transaction manager enabled causes transaction conflict #7217

Repeatable {in, up}sert with memtx transaction manager enabled causes transaction conflict #7217

CuriousGeorgiy commented May 30, 2022

Repeatable {in, up}sert with memtx transaction manager enabled causes transaction conflict #7217

Repeatable {in, up}sert with memtx transaction manager enabled causes transaction conflict #7217

Comments

CuriousGeorgiy commented May 30, 2022