Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make ingest external file backward compatible #1783

Closed
wants to merge 3 commits into from
Closed

Make ingest external file backward compatible #1783

wants to merge 3 commits into from

Conversation

shuzhang1989
Copy link
Contributor

No description provided.

@shuzhang1989 shuzhang1989 mentioned this pull request Jan 18, 2017
Closed
IslamAbdelRahman
IslamAbdelRahman reviewed Jan 18, 2017
View reviewed changes
db/external_sst_file_ingestion_job.cc
@@ -291,24 +291,26 @@ Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
file_to_ingest->version = DecodeFixed32(version_iter->second.c_str());

auto seqno_iter = uprops.find(ExternalSstFilePropertyNames::kGlobalSeqno);
if (file_to_ingest->version == 2) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep all these checks for V2, but add a special condition for V1.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

IslamAbdelRahman
IslamAbdelRahman reviewed Jan 18, 2017
View reviewed changes
db/external_sst_file_ingestion_job.cc
return Status::Corruption("Was not able to find file global seqno field");
}
} else {
return Status::InvalidArgument("external file version is not supported");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should keep this as well (let's say we are ingesting V3 or V4, we should fail)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

@facebook-github-bot
Copy link
Contributor

@shuzhang1989 updated the pull request - view changes

@shuzhang1989
Copy link
Contributor Author

@IslamAbdelRahman updated

@facebook-github-bot
Copy link
Contributor

@shuzhang1989 updated the pull request - view changes

IslamAbdelRahman
IslamAbdelRahman reviewed Jan 19, 2017
View reviewed changes
Copy link
Contributor

@IslamAbdelRahman IslamAbdelRahman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM .. just some minor comments

db/external_sst_file_ingestion_job.cc
ingestion_options_.allow_global_seqno == true)) {
return Status::InvalidArgument(
"external file's version number is not correct");
} else if (supported_versions.find(file_to_ingest->version) ==
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's just keep this to be

else {
   return Status::InvalidArgument("External SST file version is not supported");
}

I don't think that we need the supported_versions list

db/external_sst_file_ingestion_job.cc
} else if (file_to_ingest->version == 1 &&
(ingestion_options_.allow_blocking_flush == true ||
ingestion_options_.allow_global_seqno == true)) {
return Status::InvalidArgument(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's add an assert like this

// SST file V1 should not have global seqno field
assert(seqno_iter == uprops.end())

db/external_sst_file_ingestion_job.cc
(ingestion_options_.allow_blocking_flush == true ||
ingestion_options_.allow_global_seqno == true)) {
return Status::InvalidArgument(
"external file's version number is not correct");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's update the error message to be something like this

"External SST file V1 does not support global seqno"

@facebook-github-bot
Copy link
Contributor

@shuzhang1989 updated the pull request - view changes

IslamAbdelRahman
IslamAbdelRahman approved these changes Jan 19, 2017
View reviewed changes
Copy link
Contributor

@IslamAbdelRahman IslamAbdelRahman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! will land after tests pass

@facebook-github-bot
Copy link
Contributor

@IslamAbdelRahman has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@shuzhang1989 shuzhang1989 mentioned this pull request Feb 15, 2017
Closed
facebook-github-bot pushed a commit that referenced this pull request Feb 17, 2017
@shuzhang1989 @facebook-github-bot
Allow adding external v1 sst file with no global seqno support
756c592 
Summary:
This is a follow up fix for #1783. After it, we should be able to ingest external v1 sst files with no global seqno field.
Closes #1874

Differential Revision: D4576194

Pulled By: IslamAbdelRahman

fbshipit-source-id: 5b34a3e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@IslamAbdelRahman IslamAbdelRahman IslamAbdelRahman approved these changes

Assignees
No one assigned
Labels
CLA Signed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@shuzhang1989 @facebook-github-bot @IslamAbdelRahman