Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix compiler warning and base blob path case for searchable snapshots #6558

Merged
merged 1 commit into from
Mar 8, 2023

Conversation

samuel-oci
Copy link
Contributor

@samuel-oci samuel-oci commented Mar 6, 2023

Description

Fixing base blob path scenario for searchable snapshots

  1. Adding base blob path configuration for FS directory
  2. Add base blob path scenario in searchable snapshot tests and fix bug
  3. Fix compiler warnings

Issues Resolved

Closes #6557

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • [] New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2023

Gradle Check (Jenkins) Run Completed with:

@kotwanikunal
Copy link
Member

@samuel-oci Thanks for the contribution.
Can you please run ./gradlew spotlessApply to fix spotless errors and ./gradlew precommit to validate other checks?

More on that here.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.smoketest.SmokeTestMultiNodeClientYamlTestSuiteIT.test {yaml=pit/10_basic/Delete all}
      1 org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testIndexCreateBlockIsRemovedWhenAnyNodesNotExceedHighWatermarkWithAutoReleaseEnabled
      1 org.opensearch.cluster.allocation.AwarenessAllocationIT.testThreeZoneOneReplicaWithForceZoneValueAndLoadAwareness

@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2023

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      1 org.opensearch.search.SearchWeightedRoutingIT.testSearchAggregationWithNetworkDisruption_FailOpenEnabled

@samuel-oci
Copy link
Contributor Author

@samuel-oci Thanks for the contribution. Can you please run ./gradlew spotlessApply to fix spotless errors and ./gradlew precommit to validate other checks?

More on that here.

Hi @kotwanikunal thanks for the feedback, done.
I do notice that there are some flakey IT tests that are not related to this change, but the gradle check seems to be passing.
Let me know if anything else is needed.

@kotwanikunal
Copy link
Member

@samuel-oci This LGTM!
@andrross Can you please have a look as well?

@kotwanikunal kotwanikunal added backport 2.x Backport to 2.x branch skip-changelog labels Mar 7, 2023
Copy link
Member

@andrross andrross left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thanks @samuel-oci!

I actually think it makes sense to add a changelog entry here for "Add 'base_path' setting to File System Repository" since this is a user-facing feature. @kotwanikunal what do you think?

@@ -193,7 +193,7 @@ public void testAllocationWithDisruption() throws Exception {

/**
* Utility method which ensures cluster has balanced primary shard distribution across a single index.
* @throws Exception
* @throws Exception exception
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just an FYI, if you've been getting javadoc warnings/failures in the tests, it might be due to building with JDK 14. The javadoc rules are supposed to be ignored for tests, but JDK-14 doesn't seem to do the right thing, though it has been end-of-life for quite awhile now.

@kotwanikunal
Copy link
Member

Looks great, thanks @samuel-oci!

I actually think it makes sense to add a changelog entry here for "Add 'base_path' setting to File System Repository" since this is a user-facing feature. @kotwanikunal what do you think?

Ahh. Yep. I think since we have a new setting for FsRepository, an entry makes sense.
@samuel-oci You can follow the CHANGELOG guide here - https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#changelog
Please add an entry into the Unreleased 2.x section since this will be backported to 2.x

Signed-off-by: Samuel Herman <sherman8915@gmail.com>
@github-actions
Copy link
Contributor

github-actions bot commented Mar 7, 2023

Gradle Check (Jenkins) Run Completed with:

@kotwanikunal
Copy link
Member

Gradle Check (Jenkins) Run Completed with:

Unrelated failure due to heap space usage.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 8, 2023

Gradle Check (Jenkins) Run Completed with:

@samuel-oci
Copy link
Contributor Author

Thanks for your comments @kotwanikunal @andrross , added a CHANGELOG entry.

@andrross andrross merged commit 8c0e5d0 into opensearch-project:main Mar 8, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-6558-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 8c0e5d0c680f51ecea82468c622508bda510cc26
# Push it to GitHub
git push --set-upstream origin backport/backport-6558-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-6558-to-2.x.

@samuel-oci samuel-oci deleted the fix-blobpath-bug branch March 8, 2023 17:07
@kotwanikunal kotwanikunal mentioned this pull request Mar 8, 2023
6 tasks
@kotwanikunal kotwanikunal added backport 2.x Backport to 2.x branch and removed backport 2.x Backport to 2.x branch labels Mar 8, 2023
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-6558-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 8c0e5d0c680f51ecea82468c622508bda510cc26
# Push it to GitHub
git push --set-upstream origin backport/backport-6558-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-6558-to-2.x.

kotwanikunal pushed a commit to kotwanikunal/OpenSearch that referenced this pull request Mar 8, 2023
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
Co-authored-by: Samuel Herman <samuelherman@Samuels-MBP.MG8702>
(cherry picked from commit 8c0e5d0)
@kotwanikunal
Copy link
Member

Backport: #6584

kotwanikunal pushed a commit to kotwanikunal/OpenSearch that referenced this pull request Mar 8, 2023
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
Co-authored-by: Samuel Herman <samuelherman@Samuels-MBP.MG8702>
(cherry picked from commit 8c0e5d0)
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
kotwanikunal added a commit that referenced this pull request Mar 8, 2023
* Implement equals/hashcode for named DocValueFormat inner classes (#6357)

Signed-off-by: Bryan Burkholder <bburkholder@slack-corp.com>
(cherry picked from commit 5a7fd4f)
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>

* Fix compiler warning and base blob path case (#6558)

Signed-off-by: Samuel Herman <sherman8915@gmail.com>
Co-authored-by: Samuel Herman <samuelherman@Samuels-MBP.MG8702>
(cherry picked from commit 8c0e5d0)
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>

---------

Signed-off-by: Bryan Burkholder <bburkholder@slack-corp.com>
Signed-off-by: Kunal Kotwani <kkotwani@amazon.com>
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
Co-authored-by: Bryan Burkholder <771133+bryanlb@users.noreply.github.com>
Co-authored-by: samuel-oci <97131656+samuel-oci@users.noreply.github.com>
Co-authored-by: Samuel Herman <samuelherman@Samuels-MBP.MG8702>
mingshl pushed a commit to mingshl/OpenSearch-Mingshl that referenced this pull request Mar 24, 2023
Signed-off-by: Samuel Herman <sherman8915@gmail.com>
Co-authored-by: Samuel Herman <samuelherman@Samuels-MBP.MG8702>
Signed-off-by: Mingshi Liu <mingshl@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] blob base path not taken into account in searchable snapshot implementation
3 participants