Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tuple missing in secondary index if DML races with DDL #6263

Closed
locker opened this issue Jul 28, 2021 · 0 comments · Fixed by #6408
Closed

Tuple missing in secondary index if DML races with DDL #6263

locker opened this issue Jul 28, 2021 · 0 comments · Fixed by #6408
Assignees
Labels
bug Something isn't working ddl memtx mvcc
Milestone

Comments

@locker
Copy link
Member

locker commented Jul 28, 2021

Version: 2.9.0-166-g71a67913af75

The following script

fiber = require('fiber')
yaml = require('yaml')

box.cfg{memtx_use_mvcc_engine = true}

s = box.schema.space.create('test')
s:create_index('pk')

ch1 = fiber.channel()
ch2 = fiber.channel()

fiber.create(function()
    box.begin()
    s:insert{1, 1}
    ch1:get()
    box.commit()
    ch2:put(true)
end)

s:create_index('sk', {parts = {{2, 'unsigned'}}})

ch1:put(true)
ch2:get()

print('Primary:')
print(yaml.encode(s.index.pk:select()))

print('Secondary:')
print(yaml.encode(s.index.sk:select()))

os.exit(0)

prints the following

Primary:
---
- [1, 1]
...

Secondary:
--- []
...

That is, a tuple present in the space is not visible via the secondary index.

Vinyl fixes this problem by aborting all pending transactions before proceeding to building an index.

The same bug probably affects space format change. Please check.

@locker locker added bug Something isn't working memtx mvcc labels Jul 28, 2021
@locker locker added the ddl label Jul 28, 2021
@locker locker mentioned this issue Jul 28, 2021
@kyukhin kyukhin added this to the 2.8.3 milestone Aug 26, 2021
@Korablev77 Korablev77 added the 5sp label Aug 26, 2021
Korablev77 added a commit that referenced this issue Aug 31, 2021
There was a bug that led to dirty read after space alter. For the
simplicity sake imagine following setup:

-- space 's' is empty
tx1:begin()
tx1('s:replace{2}')
s:alter({format = format})
s:select{}

Last select returns tuple {2}, however transaction tx1 hasn't been
committed. This happens due to the fact that during alter operation we
create new space, swap all unchanged parts of old space and then delete
old space. During removal of old space we also clean-up all stories
related to it. In turn story destruction may make dirty tuple clean in
case it remains in the index. In the previous implementation there was
no removal of uncommitted tuples from corresponding indexes. So let's
rollback all changes happened to the space right in time of alter. It is
legal since DDL operation anyway aborts ALL other transactions.

Closes #6318
Closes #6263
Korablev77 added a commit that referenced this issue Sep 1, 2021
There was a bug that led to dirty read after space alter. For the
simplicity sake imagine following setup:

-- space 's' is empty
tx1:begin()
tx1('s:replace{2}')
s:alter({format = format})
s:select{}

Last select returns tuple {2}, however transaction tx1 hasn't been
committed. This happens due to the fact that during alter operation we
create new space, swap all unchanged parts of old space and then delete
old space. During removal of old space we also clean-up all stories
related to it. In turn story destruction may make dirty tuple clean in
case it remains in the index. In the previous implementation there was
no removal of uncommitted tuples from corresponding indexes. So let's
rollback all changes happened to the space right in time of alter. It is
legal since DDL operation anyway aborts ALL other transactions.

Closes #6318
Closes #6263
Korablev77 added a commit that referenced this issue Sep 1, 2021
There was a bug that led to dirty read after space alter. For the
simplicity sake imagine following setup:

-- space 's' is empty
tx1:begin()
tx1('s:replace{2}')
s:alter({format = format})
s:select{}

Last select returns tuple {2}, however transaction tx1 hasn't been
committed. This happens due to the fact that during alter operation we
create new space, swap all unchanged parts of old space and then delete
old space. During removal of old space we also clean-up all stories
related to it. In turn story destruction may make dirty tuple clean in
case it remains in the index. In the previous implementation there was
no removal of uncommitted tuples from corresponding indexes. So let's
rollback all changes happened to the space right in time of alter. It is
legal since DDL operation anyway aborts ALL other transactions.

Closes #6318
Closes #6263
@alyapunov alyapunov removed their assignment Oct 7, 2021
Korablev77 added a commit that referenced this issue Oct 7, 2021
There was a bug that led to dirty read after space alter. For the
simplicity sake imagine following setup:

-- space 's' is empty
tx1:begin()
tx1('s:replace{2}')
s:alter({format = format})
s:select{}

Last select returns tuple {2}, however transaction tx1 hasn't been
committed. This happens due to the fact that during alter operation we
create new space, swap all unchanged parts of old space and then delete
old space. During removal of old space we also clean-up all stories
related to it. In turn story destruction may make dirty tuple clean in
case it remains in the index. In the previous implementation there was
no removal of uncommitted tuples from corresponding indexes. So let's
rollback all changes happened to the space right in time of alter. It is
legal since DDL operation anyway aborts ALL other transactions.

Closes #6318
Closes #6263
Korablev77 added a commit that referenced this issue Oct 7, 2021
There was a bug that led to dirty read after space alter. For the
simplicity sake imagine following setup:

-- space 's' is empty
tx1:begin()
tx1('s:replace{2}')
s:alter({format = format})
s:select{}

Last select returns tuple {2}, however transaction tx1 hasn't been
committed. This happens due to the fact that during alter operation we
create new space, swap all unchanged parts of old space and then delete
old space. During removal of old space we also clean-up all stories
related to it. In turn story destruction may make dirty tuple clean in
case it remains in the index. In the previous implementation there was
no removal of uncommitted tuples from corresponding indexes. So let's
rollback all changes happened to the space right in time of alter. It is
legal since DDL operation anyway aborts ALL other transactions.

Closes #6318
Closes #6263

(cherry picked from commit ce5752c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working ddl memtx mvcc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants