Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add create or replace stage support #14548

Merged
merged 3 commits into from Feb 1, 2024

Conversation

lichuang
Copy link
Collaborator

@lichuang lichuang commented Feb 1, 2024

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feat: add create or replace stage support

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Feb 1, 2024
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 2 of 13 files at r1, all commit messages.
Reviewable status: 2 of 13 files reviewed, 1 unresolved discussion (waiting on @lichuang)


src/query/management/src/stage/stage_mgr.rs line 107 at r1 (raw file):

                &key,
                serialize_struct(&info, ErrorCode::IllegalUserStageFormat, || "")?,
            )];

Why is it written as a transaction here? Wouldn't a simple upsert be enough? For CreateIfNotExists, it would be MatchSeq::Exact(0); and for CreateOrReplace, it would be MatchSeq::GE(0).

Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 2 of 13 files reviewed, 1 unresolved discussion (waiting on @lichuang)


src/query/management/src/stage/stage_mgr.rs line 106 at r2 (raw file):

        self.kv_api
            .upsert_kv(UpsertKVReq::new(&key, seq, val, None))
            .await?;

The result of the upsert must be checked to see if it succeeded.
And the get() before upsert is unnecessary.

Suggestion:

let seq = match create_option {
    CreateOption::CreateIfNotExists(_) => MatchSeq::Exact(0),
    CreateOption::CreateOrReplace => MatchSeq::GE(0),
};

let res = self.kv_api
    .upsert_kv(UpsertKVReq::new(&key, seq, val, None))
    .await?;

match create_option {
    CreateOption::CreateIfNotExists(false) => {
        if res.prev.is_some() {
            return Err(ErrorCode::StageAlreadyExists(format!(
                "Stage '{}' already exists.",
                info.stage_name
            )));
        }
    },
    _ => {}
}

Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 7 of 13 files at r1, 1 of 1 files at r3, all commit messages.
Reviewable status: 10 of 13 files reviewed, all discussions resolved

@BohuTANG BohuTANG merged commit 7493b98 into datafuselabs:main Feb 1, 2024
71 of 72 checks passed
@lichuang
Copy link
Collaborator Author

lichuang commented Feb 2, 2024

@soyeric128

@lichuang lichuang deleted the replace_or_create_stage branch February 8, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: add create or replace stage support
3 participants