Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unification of is_deleted for RMT with LWD-based _row_exists #63440

Open
UnamedRus opened this issue May 7, 2024 · 2 comments
Open

unification of is_deleted for RMT with LWD-based _row_exists #63440

UnamedRus opened this issue May 7, 2024 · 2 comments
Labels
feature minor Priority: minor st-discussion The story requires discussion /research / expert help / design & decomposition before will be taken

Comments

@UnamedRus
Copy link
Contributor

UnamedRus commented May 7, 2024

Use case

Adjust _row_exists for better support in RMT:

  1. Allow to insert _row_exists column in INSERT statement.
  2. _row_exists = 0, effectively mark previous versions of row also "deleted" during SELECT FINAL query.
  3. Do not remove _row_exists rows from RMT parts during merge (unconditionally, but in some cases they can be removed)

For second, it may require reorder of reading steps:

  1. do not "filter out" _row_exists=0 row during Vertical FINAL step and keep it, if it's last version
  2. drop it from final result set.

If, this will be implemented, it does mean that RMT will have LWD and ultra-LWD (without costly mutation), and reduce demand for is_deleted support.

@alexey-milovidov alexey-milovidov added minor Priority: minor st-discussion The story requires discussion /research / expert help / design & decomposition before will be taken labels May 7, 2024
@alexey-milovidov
Copy link
Member

This will be difficult to understand by users.

@UnamedRus
Copy link
Contributor Author

Interesting that Apache Doris have concept of visible for user DORIS_DELETE_SIGN (basically, what is_deleted currently does in ReplacingMergeTree)
https://doris.apache.org/docs/data-operate/delete/batch-delete-manual#usage-examples

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature minor Priority: minor st-discussion The story requires discussion /research / expert help / design & decomposition before will be taken
Projects
None yet
Development

No branches or pull requests

2 participants