Skip to content

Fix insert&delete cases dead data.#11773

Merged
JackieTien97 merged 2 commits intoapache:masterfrom
ColinLeeo:colin_fix_deletion_parallel
Jan 4, 2024
Merged

Fix insert&delete cases dead data.#11773
JackieTien97 merged 2 commits intoapache:masterfrom
ColinLeeo:colin_fix_deletion_parallel

Conversation

@ColinLeeo
Copy link
Copy Markdown
Collaborator

@ColinLeeo ColinLeeo commented Dec 22, 2023

In this PR:

  1. remove useless write lock on schema cache when execute deletion
  2. coordinator will acquire a read lock on schema cache, which will block deletion's cache invalidation.
  3. solves the DEAD data problem: these data have no metadata but only data.

@ColinLeeo ColinLeeo force-pushed the colin_fix_deletion_parallel branch from 24c0793 to 34c53fa Compare December 22, 2023 07:03
@ColinLeeo
Copy link
Copy Markdown
Collaborator Author

ColinLeeo commented Dec 22, 2023

You can reproduce the problem using the steps below:
add code at the end of function validateSchema()

    try {
      Thread.sleep(10 * 1000);
    } catch (InterruptedException e) {
      // nothing
    }

run create and insert in one cli:

CREATE TIMESERIES root.db.d1.s1 WITH DATATYPE=INT32, ENCODING=PLAIN
INSERT INTO root.db.d1(timestamp,s1) values(1,1)

run delete on another cli:

delete timeseries  root.db.d1.s1

Without this patch, Insert operation will insert data into root.db.d1.s1 that has been deleted in another cli.
While with this patch, delete operation will wait insert operation finish so it can delete timeseries safely.

@ColinLeeo ColinLeeo force-pushed the colin_fix_deletion_parallel branch from 34c53fa to a90f71b Compare December 22, 2023 09:18
@ColinLeeo ColinLeeo force-pushed the colin_fix_deletion_parallel branch from b1f8080 to e8791d7 Compare January 3, 2024 09:59
@JackieTien97 JackieTien97 merged commit 491246d into apache:master Jan 4, 2024
@ColinLeeo ColinLeeo changed the title fix insert&delete cases dead data. Fix insert&delete cases dead data. Feb 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants