Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deferred DELETE isn't produced on transaction commit #4248

Closed
locker opened this issue May 24, 2019 · 3 comments
Closed

Deferred DELETE isn't produced on transaction commit #4248

locker opened this issue May 24, 2019 · 3 comments
Assignees
Labels
bug Something isn't working vinyl
Milestone

Comments

@locker
Copy link
Member

locker commented May 24, 2019

Tarantool version:

Reproduced using 2.2.0-343-gd07e3ac2. Both 2.1 and 1.10 are affected.

Bug description:

Under certain conditions a deferred DELETE statement isn't generated when a transaction is committed. This may result in garbage statements piling up in secondary index runs.

Steps to reproduce:

Run the following script in an empty directory.

box.cfg{}

s = box.schema.space.create('test', {engine = 'vinyl'})
pk = s:create_index('pk')
sk = s:create_index('sk', {parts = {2, 'unsigned'}})

s:insert{1, 1}

box.begin()
s:replace{1, 2}
s:update(1, {{'=', 2, 3}})
box.commit()

box.snapshot()
os.exit()

Now take a look at the written run files:

$ tarantoolctl cat */*/*.run
Processing file '512/0/00000000000000000006.run'
---
HEADER:
  lsn: 7
  type: INSERT
BODY:
  tuple: [1, 3]
---
HEADER:
  type: ROWINDEX
BODY:
  row_index: "\0\0\0\0"
...

Processing file '512/1/00000000000000000004.run'
---
HEADER:
  lsn: 5
  type: INSERT
BODY:
  tuple: [1, 1]
---
HEADER:
  lsn: 7
  type: INSERT
BODY:
  tuple: [3, 1]
---
HEADER:
  type: ROWINDEX
BODY:
  row_index: "\0\0\0\0\0\0\0\n"
...

Note statement INSERT{1,1} in the secondary index run. It isn't present in the primary index hence can't be purged.

@locker locker added bug Something isn't working vinyl labels May 24, 2019
@locker locker added this to the 1.10.4 milestone May 24, 2019
@locker locker self-assigned this May 24, 2019
locker added a commit that referenced this issue May 24, 2019
Even if a statement isn't marked as VY_STMT_DEFERRED_DELETE, e.g. it's
a REPLACE produced by an UPDATE request, it may overwrite a statement in
the transaction write set that is marked so, for instance:

  s = box.schema.space.create('test', {engine = 'vinyl'})
  pk = s:create_index('pk')
  sk = s:create_index('sk', {parts = {2, 'unsigned'}})

  s:insert{1, 1}

  box.begin()
  s:replace{1, 2}
  s:update(1, {{'=', 2, 3}})
  box.commit()

If we don't mark REPLACE{3,1} produced by the update operatoin with
VY_STMT_DEFERRED_DELETE flag, we will never generate a DELETE statement
for INSERT{1,1}. That is, we must inherit the flag from the overwritten
statement when we insert a new one into a write set.

Closes #4248
@locker locker closed this as completed in b54433d May 27, 2019
locker added a commit that referenced this issue May 27, 2019
Even if a statement isn't marked as VY_STMT_DEFERRED_DELETE, e.g. it's
a REPLACE produced by an UPDATE request, it may overwrite a statement in
the transaction write set that is marked so, for instance:

  s = box.schema.space.create('test', {engine = 'vinyl'})
  pk = s:create_index('pk')
  sk = s:create_index('sk', {parts = {2, 'unsigned'}})

  s:insert{1, 1}

  box.begin()
  s:replace{1, 2}
  s:update(1, {{'=', 2, 3}})
  box.commit()

If we don't mark REPLACE{3,1} produced by the update operatoin with
VY_STMT_DEFERRED_DELETE flag, we will never generate a DELETE statement
for INSERT{1,1}. That is, we must inherit the flag from the overwritten
statement when we insert a new one into a write set.

Closes #4248

(cherry picked from commit b54433d)
locker added a commit that referenced this issue May 27, 2019
Even if a statement isn't marked as VY_STMT_DEFERRED_DELETE, e.g. it's
a REPLACE produced by an UPDATE request, it may overwrite a statement in
the transaction write set that is marked so, for instance:

  s = box.schema.space.create('test', {engine = 'vinyl'})
  pk = s:create_index('pk')
  sk = s:create_index('sk', {parts = {2, 'unsigned'}})

  s:insert{1, 1}

  box.begin()
  s:replace{1, 2}
  s:update(1, {{'=', 2, 3}})
  box.commit()

If we don't mark REPLACE{3,1} produced by the update operatoin with
VY_STMT_DEFERRED_DELETE flag, we will never generate a DELETE statement
for INSERT{1,1}. That is, we must inherit the flag from the overwritten
statement when we insert a new one into a write set.

Closes #4248

(cherry picked from commit b54433d)
avtikhon pushed a commit that referenced this issue May 29, 2019
Even if a statement isn't marked as VY_STMT_DEFERRED_DELETE, e.g. it's
a REPLACE produced by an UPDATE request, it may overwrite a statement in
the transaction write set that is marked so, for instance:

  s = box.schema.space.create('test', {engine = 'vinyl'})
  pk = s:create_index('pk')
  sk = s:create_index('sk', {parts = {2, 'unsigned'}})

  s:insert{1, 1}

  box.begin()
  s:replace{1, 2}
  s:update(1, {{'=', 2, 3}})
  box.commit()

If we don't mark REPLACE{3,1} produced by the update operatoin with
VY_STMT_DEFERRED_DELETE flag, we will never generate a DELETE statement
for INSERT{1,1}. That is, we must inherit the flag from the overwritten
statement when we insert a new one into a write set.

Closes #4248
@Korablev77
Copy link
Contributor

Fix was pushed to master, but forgotten to be pushed to 1.10.

@kostja
Copy link
Contributor

kostja commented Apr 10, 2020

Please cherry-pick?

@Korablev77
Copy link
Contributor

@kostja already done:9fdeccb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working vinyl
Projects
None yet
Development

No branches or pull requests

3 participants