-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deprecate setting 'reindex.remote.whitelist' and introduce the alternative setting 'reindex.remote.allowlist' #2221
Deprecate setting 'reindex.remote.whitelist' and introduce the alternative setting 'reindex.remote.allowlist' #2221
Conversation
…emote.whitelist Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Can one of the admins verify this patch? |
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Tianli Feng <ftianli@amazon.com>
} | ||
|
||
public void testSettingGetValueWithFallback() { | ||
Settings settings = Settings.builder().put("reindex.remote.whitelist", "127.0.0.1:*").build(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What happens if both "reindex.remote.allowlist" and "reindex.remote.whitelist" are specified but they define different values?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @andrross , thanks for your review! I have got the concern here, I know it will make the user confused when using the 2 different expression for this setting. The confusion is little when the setting is static - can only be configured through opensearch.yml
file, but it will be larger when the setting is dynamic - can be configured in both "cluster setting API" and opensearch.yml
file.
All usages of the "whitelist" setting in the code base have been changed to the "allowlist" setting, so I think the value of the "whitelist" will be ignored, once "allowlist" is configured.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The unit test here also shows the behavior,
https://github.com/opensearch-project/job-scheduler/pull/20/files#diff-2d466c782fd61b8154ef2e4ec067b3d528cc92c9ce6cd4d6401475568077abf1R87
Although they can have different value, with my code change, value of the old "whitelist" setting will not be taken anywhere.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was actually a bit of a leading question :) Can you create a unit test to demonstrate/validate this behavior? For example, if you set both settings to different values and use TransportReindexAction.REMOTE_CLUSTER_ALLOWLIST
, only the "allowlist" value should be returned because the fallback setting should not be used.
Relatedly, can you make the TransportReindexAction.REMOTE_CLUSTER_WHITELIST
constant private? I believe it should not be directly used anywhere and the only access to that setting is via the fallback mechanism of TransportReindexAction.REMOTE_CLUSTER_ALLOWLIST
. Please correct me if I'm wrong.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your suggestion! I probably can just modify the test case above: testSettingGetValue()
to give a different value of the "whitelist" setting, and then get both of the setting value by calling TransportReindexAction.REMOTE_CLUSTER_*LIST
.
I just noticed there are deprecated setting be set as private variable, such as https://github.com/opensearch-project/OpenSearch/blob/1.2.4/server/src/main/java/org/opensearch/node/Node.java#L229, this is a good idea to me! I will learn more impact about it.
Currently the "whitelist" setting is still added into the Settings list of the plugin (
OpenSearch/modules/reindex/src/main/java/org/opensearch/index/reindex/ReindexPlugin.java
Line 135 in c8dfd68
settings.add(TransportReindexAction.REMOTE_CLUSTER_WHITELIST); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm worried that if removing the Line 135 settings.add(TransportReindexAction.REMOTE_CLUSTER_WHITELIST);
, any custom value of the "whitelist" setting will be ignored, which leads "allowlist" setting fallback to the default value of "whitelist" setting. Let me validate through the unit test.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand. The desired behavior is that if 'allowlist' is specified then 'whitelist' should be ignored. If only 'whitelist' is specified then it will be used. Do we still get that behavior if your line 135?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @andrross, I will do an experiment in a real cluster to validate whether the Line 135 is necessary or not. I'm only afraid that without putting that setting in the "known settings" list, there will be an error on start up. (https://www.elastic.co/guide/en/elasticsearch/reference/7.15/archived-settings.html)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, please do that quick experiment because it would make the code much cleaner (and the tests simpler) to just have that fallback setting as a private field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assembled an linux tar distribution of OpenSearch, after removing "whitelist" setting from the Settings list of Reindex module. In the distribution, I configured reindex.remote.whitelist
in opensearch.yml
, then start the node, but the error appears.
But I can restrict the use of constant TransportReindexAction.REMOTE_CLUSTER_WHITELIST
in the same package.
[2022-02-24T23:10:08,579][ERROR][o.o.b.OpenSearchUncaughtExceptionHandler] [ip-172-31-15-168] uncaught exception in thread [main]
org.opensearch.bootstrap.StartupException: java.lang.IllegalArgumentException: unknown setting [reindex.remote.whitelist] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:182) ~[opensearch-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:169) ~[opensearch-2.0.0-SNAPSHOT.jar:2.0.0-SNAPSHOT]
...
uncaught exception in thread [main]
java.lang.IllegalArgumentException: unknown setting [reindex.remote.whitelist] please check that any required plugins are installed, or check the breaking changes documentation for removed settings
at org.opensearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:589)
...
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:135)
at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:101)
…ot a value Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Tianli Feng <ftianli@amazon.com>
@@ -101,7 +101,7 @@ static void checkRemoteWhitelist(CharacterRunAutomaton whitelist, RemoteInfo rem | |||
if (whitelist.run(check)) { | |||
return; | |||
} | |||
String whiteListKey = TransportReindexAction.REMOTE_CLUSTER_WHITELIST.getKey(); | |||
String whiteListKey = TransportReindexAction.REMOTE_CLUSTER_ALLOWLIST.getKey(); | |||
throw new IllegalArgumentException('[' + check + "] not whitelisted in " + whiteListKey); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we change this exception message?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since PR #2178 is merged, the "whitelisted" in the exception messages have changed to "allowlisted".
@@ -308,7 +308,7 @@ | |||
--- | |||
"unwhitelisted remote host fails": | |||
- do: | |||
catch: /\[badremote:9200\] not whitelisted in reindex.remote.whitelist/ | |||
catch: /\[badremote:9200\] not whitelisted in reindex.remote.allowlist/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will need to be updated if you change the exception message, I think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as above. (Since PR #2178 is merged, the "whitelisted" in the exception messages have changed to "allowlisted") 😁
@@ -308,7 +308,7 @@ | |||
--- | |||
"unwhitelisted remote host fails": |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this test description be changed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch! Thank you. This one is missed in PR #2178, I changed it in this PR.
…HITELIST Signed-off-by: Tianli Feng <ftianli@amazon.com>
ReindexPlugin plugin; | ||
|
||
@Before | ||
public void setup() { | ||
this.plugin = new ReindexPlugin(); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nitpick, but I think you can replace all this with just:
private final ReindexPlugin plugin = new ReindexPlugin();
There's generally no need for a @Before
method unless you need more complex logic than field assignment. (One note about JUnit that often trips people up...the test framework creates a brand new instance of the test class before each individual test run. That mean no instance state is kept between test runs, which is why the simple inline assignment functions the same as assigning fields in a @Before
method.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much for sharing this knowledge to me! 👍👍 I totally had no idea with it. 😲 I will make the change.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
JUnit5 allows you to specify the test instance lifecycle, though the create-new-instance-for-each-test-method behavior is the default and matches the behavior of all previous versions of JUnit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for telling me the background theory! Hope more people notice it. 😄 I see, since the default lifecycle is "per method", so any class variable will be initialized before calling any method.
(Though in this file, the variable only used in one method 😅)
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Test failure in Log 2821 has been reported in issue #1908 |
start gradle check |
start gradle check |
# Conflicts: # modules/reindex/build.gradle # modules/reindex/src/main/java/org/opensearch/index/reindex/ReindexValidator.java # modules/reindex/src/test/java/org/opensearch/index/reindex/ReindexFromRemoteWhitelistTests.java # modules/reindex/src/test/java/org/opensearch/index/reindex/ReindexFromRemoteWithAuthTests.java # modules/reindex/src/test/java/org/opensearch/index/reindex/RetryTests.java # modules/reindex/src/yamlRestTest/resources/rest-api-spec/test/reindex/20_validation.yml
… api testing. Signed-off-by: Tianli Feng <ftianli@amazon.com>
In log 3129:
Reported in issue #2359 In log 3131:
Reported in issue #1746 |
start gradle check |
Will need documentation. I presume we do not want to backport this to 1.3? |
…ative setting 'reindex.remote.allowlist' (#2221) * Add setting reindex.remote.allowlist, and deprecate setting reindex.remote.whitelist Signed-off-by: Tianli Feng <ftianli@amazon.com> * Add unit test for renaming the setting reindex.remote.allowlist Signed-off-by: Tianli Feng <ftianli@amazon.com> * Remove system.out.println() Signed-off-by: Tianli Feng <ftianli@amazon.com> * Adjust format by spotlessApply task Signed-off-by: Tianli Feng <ftianli@amazon.com> * Replace REMOTE_CLUSTER_WHITELIST with REMOTE_CLUSTER_ALLOWLIST Signed-off-by: Tianli Feng <ftianli@amazon.com> * Add a unit test to test final setting value when both settings have got a value Signed-off-by: Tianli Feng <ftianli@amazon.com> * Rename the unit test class name Signed-off-by: Tianli Feng <ftianli@amazon.com> * Remove the Access modifiers public from the constant REMOTE_CLUSTER_WHITELIST Signed-off-by: Tianli Feng <ftianli@amazon.com> * Initialize ReindexPlugin without using the @before method Signed-off-by: Tianli Feng <ftianli@amazon.com> * Rename 'unwhitelisted' to 'unallowlisted' in a yml file used for REST api testing. Signed-off-by: Tianli Feng <ftianli@amazon.com> (cherry picked from commit 63c75d1)
Hi @dblock , thank you for merging this PR! No problem, I will track the progress on the documentation. It can be backported to 1.3 version, since this is backwards compatibility, but if the time is too limited, it can be in 2.x only. |
I think it's a product question where we may have wanted to group changes related to nomenclature in 2.0, so @elfisher do we want to deprecate this setting in 1.3 and remove it in 2.0 or 3.0, or deprecate it in 2.0 and remove it in 3.0? |
Hi @dblock , I understand this concern, it will make our roadmap clearer if grouping the inclusive language changes on REST API and settings in major version 2.0. I agree to keep the change in 2.x version only. |
This includes the following: commit 9cfa395 Author: Kartik <gkart@amazon.com> Date: Thu Mar 10 10:12:17 2022 -0800 Remove the IndexCommitRef class (opensearch-project#2421) This inner class is no longer required because its functionality has been moved to the generic GatedCloseable class. Signed-off-by: Kartik Ganesh <gkart@amazon.com> commit c8d8009 Author: Andriy Redko <andriy.redko@aiven.io> Date: Thu Mar 10 11:46:08 2022 -0500 Fixing bwcVersions and bwc builds (opensearch-project#2430) - adding 1.4.0 into main bwcVersions Signed-off-by: Andriy Redko <andriy.redko@aiven.io> commit fb9e150 Author: Kartik <85275476+kartg@users.noreply.github.com> Date: Wed Mar 9 12:21:09 2022 -0800 Refactoring gated and ref-counted interfaces and their implementations (opensearch-project#2396) * Reducing duplication in plugins around ref-counted releasable classes Both AmazonEc2Reference and AmazonS3Reference duplicate the same logic - a subclass of AbstractRefCounted that also implements Releasable. This change centralizes this paradigm into a AbstractRefCountedReleasable class and supports both clients via generics. It also updates all fetching implementations to use the get() method instead of client() Signed-off-by: Kartik Ganesh <gkart@amazon.com> * Introduce Reference classes for the Closeable and AutoCloseable interfaces These classes allow you to wrap a reference instance with an onClose runnable that is executed when close() is invoked. Two separate classes are needed because the close() signatures for the two interfaces are different. This change takes the first step to have implementing classes extend from these generic superclasses, before attempting to remove the subclasses entirely. The get() method is also replaced throughout the code base. Note that there is also a separate Releasable interface that has a similar access pattern, but is implemented separately. This is used in AbstractRefCountedReleasable introduced in a prior commit Signed-off-by: Kartik Ganesh <gkart@amazon.com> * More improvements and refactoring * Functionality around one-way gating is now moved to a dedicated class - OneWayGate. This replaces duplicate functionality throughout the code. * The two *Reference classes have been renamed to Gated* since that better represents their functionality * The AbstractRefCountedReleasable has been improved to no longer be abstract by accepting the shutdown hook. This removes the need for the inner class in ReleasableBytesReference, and further simplifies the plugin subclasses (these could probably be removed entirely). * Finally, unit tests have been added for some classes Signed-off-by: Kartik Ganesh <gkart@amazon.com> * Added tests for GatedCloseable Also updated the license information in GatedAutoCloseableTests Signed-off-by: Kartik Ganesh <gkart@amazon.com> * Fixing license information in new files Signed-off-by: Kartik Ganesh <gkart@amazon.com> * Added unit tests for RefCountedReleasable Signed-off-by: Kartik Ganesh <gkart@amazon.com> commit 5a9a114 Author: Nick Knize <nknize@apache.org> Date: Wed Mar 9 12:50:05 2022 -0600 [Remove] TrimUnsafeCommit logic for legacy 6.x indexes (opensearch-project#2225) * [Remove] TrimUnsafeCommit logic for legacy 6.x indexes Multiple txlog commits was introduced in legacy 7.x. Legacy 6.x indexes could therefore not have a safe commit. Since OpenSearch 2.0 is no longer compatible with legacy 6.x indexes, the logic to trim these unsafe commits is safely removed. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * fix assertion typo Signed-off-by: Nicholas Walter Knize <nknize@apache.org> * rebase and incorporate pr feedback Signed-off-by: Nicholas Walter Knize <nknize@apache.org> commit 9c679cb Author: Andriy Redko <andriy.redko@aiven.io> Date: Tue Mar 8 18:42:32 2022 -0500 MapperService has to be passed in as null for EnginePlugins CodecService constructor (opensearch-project#2177) * MapperService has to be passed in as null for EnginePlugins CodecService constructor Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Addressing code review comments Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Delayed CodecService instantiation up to the shard initialization Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Added logger (associated with shard) to CodecServiceConfig Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Refactored the EngineConfigFactory / IndexShard instantiation of the CodecService Signed-off-by: Andriy Redko <andriy.redko@aiven.io> commit a6a47e7 Author: Suraj Singh <79435743+dreamer-89@users.noreply.github.com> Date: Tue Mar 8 14:43:04 2022 -0800 Remove inclue_type_name parameter from rest api spec (opensearch-project#2410) Signed-off-by: Suraj Singh <surajrider@gmail.com> commit 044f536 Author: Daniel Doubrovkine (dB.) <dblock@dblock.org> Date: Tue Mar 8 14:48:51 2022 -0500 Set target and source compatibility to 11, required by Lucene 9. (opensearch-project#2407) * Set target and source compatibility to 11, required by Lucene 9. Signed-off-by: dblock <dblock@dblock.org> * Uncomment commented code in opensearch-project#2321 for killing child processes that uses JDK9+ ProcessInfo. Signed-off-by: dblock <dblock@dblock.org> * Set distribution checker target JDK compatibility to 11. Signed-off-by: dblock <dblock@dblock.org> * Supress processing warnings. Signed-off-by: dblock <dblock@dblock.org> commit c3712a5 Author: Nick Knize <nknize@apache.org> Date: Tue Mar 8 11:30:27 2022 -0600 [Remove] include_type_name from HLRC (opensearch-project#2397) Removes include_type_name from the high level reset client along with relevant deprecated methods in IndicesClient. All tests are updated to remove the parameter from the rest requests along with various toXContent methods that are no longer required. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> commit 63c75d1 Author: Tianli Feng <ftianli@amazon.com> Date: Tue Mar 8 08:35:36 2022 -0800 Deprecate setting 'reindex.remote.whitelist' and introduce the alternative setting 'reindex.remote.allowlist' (opensearch-project#2221) * Add setting reindex.remote.allowlist, and deprecate setting reindex.remote.whitelist Signed-off-by: Tianli Feng <ftianli@amazon.com> * Add unit test for renaming the setting reindex.remote.allowlist Signed-off-by: Tianli Feng <ftianli@amazon.com> * Remove system.out.println() Signed-off-by: Tianli Feng <ftianli@amazon.com> * Adjust format by spotlessApply task Signed-off-by: Tianli Feng <ftianli@amazon.com> * Replace REMOTE_CLUSTER_WHITELIST with REMOTE_CLUSTER_ALLOWLIST Signed-off-by: Tianli Feng <ftianli@amazon.com> * Add a unit test to test final setting value when both settings have got a value Signed-off-by: Tianli Feng <ftianli@amazon.com> * Rename the unit test class name Signed-off-by: Tianli Feng <ftianli@amazon.com> * Remove the Access modifiers public from the constant REMOTE_CLUSTER_WHITELIST Signed-off-by: Tianli Feng <ftianli@amazon.com> * Initialize ReindexPlugin without using the @before method Signed-off-by: Tianli Feng <ftianli@amazon.com> * Rename 'unwhitelisted' to 'unallowlisted' in a yml file used for REST api testing. Signed-off-by: Tianli Feng <ftianli@amazon.com> commit 65debde Author: Andriy Redko <andriy.redko@aiven.io> Date: Tue Mar 8 11:30:48 2022 -0500 Update the BWC versions (post 1.x backport) (opensearch-project#2390) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> commit 919d180 Author: Suraj Singh <79435743+dreamer-89@users.noreply.github.com> Date: Mon Mar 7 12:43:05 2022 -0800 Remove type end-points from count action (opensearch-project#2379) Signed-off-by: Suraj Singh <surajrider@gmail.com> commit 1f0361a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Mar 7 10:28:17 2022 -0800 Bump asm-commons from 5.0.4 to 9.2 in /modules/lang-expression (opensearch-project#2385) * Bump asm-commons from 5.0.4 to 9.2 in /modules/lang-expression Bumps asm-commons from 5.0.4 to 9.2. --- updated-dependencies: - dependency-name: org.ow2.asm:asm-commons dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit 09e16e3 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Mar 7 13:08:07 2022 -0500 Bump guava from 30.1.1-jre to 31.1-jre in /plugins/repository-azure (opensearch-project#2382) * Bump guava from 30.1.1-jre to 31.1-jre in /plugins/repository-azure Bumps [guava](https://github.com/google/guava) from 30.1.1-jre to 31.1-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit e1fd4b7 Author: Subhobrata Dey <sbcd90@gmail.com> Date: Mon Mar 7 08:51:49 2022 -0800 Add valuesField in PercentilesAggregationBuilder streamInput constructor (opensearch-project#2308) Signed-off-by: Subhobrata Dey <sbcd90@gmail.com> commit 4395ed5 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Mar 7 11:48:38 2022 -0500 Bump guava in /distribution/tools/upgrade-cli (opensearch-project#2383) Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 72c5d81 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Mar 7 11:48:18 2022 -0500 Bump guava in /distribution/tools/keystore-cli (opensearch-project#2384) Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 75e837d Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Mar 7 11:48:00 2022 -0500 Bump guava from 31.0.1-jre to 31.1-jre in /distribution/tools/plugin-cli (opensearch-project#2387) Bumps [guava](https://github.com/google/guava) from 31.0.1-jre to 31.1-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 3e9031f Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Mar 7 11:47:37 2022 -0500 Bump gradle-extra-configurations-plugin from 3.0.3 to 7.0.0 in /buildSrc (opensearch-project#2386) Bumps [gradle-extra-configurations-plugin](https://github.com/nebula-plugins/gradle-extra-configurations-plugin) from 3.0.3 to 7.0.0. - [Release notes](https://github.com/nebula-plugins/gradle-extra-configurations-plugin/releases) - [Changelog](https://github.com/nebula-plugins/gradle-extra-configurations-plugin/blob/main/CHANGELOG.md) - [Commits](nebula-plugins/gradle-extra-configurations-plugin@v3.0.3...v7.0.0) --- updated-dependencies: - dependency-name: com.netflix.nebula:gradle-extra-configurations-plugin dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 9224537 Author: Owais Kazi <owaiskazi19@gmail.com> Date: Fri Mar 4 16:58:13 2022 -0800 Updated the url for docker distribution (opensearch-project#2325) Signed-off-by: Owais Kazi <owaiskazi19@gmail.com> commit d5e58a2 Author: Suraj Singh <79435743+dreamer-89@users.noreply.github.com> Date: Fri Mar 4 16:40:47 2022 -0800 [Remove] Type mappings from GeoShapeQueryBuilder (opensearch-project#2322) * Remove type end-points from GeoShapeBuilder Signed-off-by: Suraj Singh <surajrider@gmail.com> * Fix integration test failures Signed-off-by: Suraj Singh <surajrider@gmail.com> commit be64af2 Author: aponb <aponb@gmx.at> Date: Sat Mar 5 01:34:09 2022 +0100 Replace exclusionary words whitelist and blacklist in the places that won't impact backwards compatibility (opensearch-project#2178) * Replace the exclusionary word whitelist with allowlist, and blacklist with denylist, in code commet and internal variable/method/class/package name. Signed-off-by: Andreas <apre@gmx.at> commit ae52008 Author: Andriy Redko <andriy.redko@aiven.io> Date: Fri Mar 4 17:44:52 2022 -0500 Fixing the --release flag usage for javac (opensearch-project#2343) (opensearch-project#2352) * Fixing the --release flag usage for javac (opensearch-project#2343) * Fixing the --release flag usage for javac Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fixing the --html5 flag usage for javadoc Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Fix java-version-checker source/target compatibility settings (opensearch-project#2354) Signed-off-by: Andriy Redko <andriy.redko@aiven.io> commit 0cc2c9b Author: Nick Knize <nknize@apache.org> Date: Fri Mar 4 13:30:43 2022 -0600 [Remove] types from PutMappingRequest (opensearch-project#2335) Remove type support from putMappingRequest, dependencies, and all tests. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> commit 729bc43 Author: Nick Knize <nknize@apache.org> Date: Fri Mar 4 10:52:24 2022 -0600 [Test-Failure] Mute TranslogPolicyIT (opensearch-project#2342) This test is slated for removal as it only applies to ancient indexes (Legacy 6.x). Muting test so bwc tests are consistent and no longer angry. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> commit 5f90227 Author: Andriy Redko <andriy.redko@aiven.io> Date: Fri Mar 4 11:12:27 2022 -0500 Add '_name' field support to score functions and provide it back in explanation response (opensearch-project#2244) * Add '_name' field support to score functions and provide it back in explanation response Signed-off-by: Andriy Redko <andriy.redko@aiven.io> * Address code review comments Signed-off-by: Andriy Redko <andriy.redko@aiven.io> commit ae14259 Author: Daniel Doubrovkine (dB.) <dblock@dblock.org> Date: Thu Mar 3 15:34:53 2022 -0500 Restore Java 8 compatibility for build tools. (opensearch-project#2300) (opensearch-project#2321) * Restore Java 8 compatibility for build tools. Signed-off-by: dblock <dblock@dblock.org> * Make source code compatible with Java 8. Signed-off-by: dblock <dblock@dblock.org> commit cb57b92 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu Mar 3 14:23:47 2022 -0500 Bump log4j-core in /buildSrc/src/testKit/thirdPartyAudit/sample_jars (opensearch-project#2281) Bumps log4j-core from 2.17.1 to 2.17.2. --- updated-dependencies: - dependency-name: org.apache.logging.log4j:log4j-core dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit cdb42ad Author: Vacha Shah <vachshah@amazon.com> Date: Thu Mar 3 10:39:27 2022 -0800 Remove Github DCO action since DCO runs via Github App now (opensearch-project#2317) Signed-off-by: Vacha Shah <vachshah@amazon.com> commit f13b951 Author: Andrey Pleskach <ples@aiven.io> Date: Wed Mar 2 23:36:09 2022 +0100 Add support of SOCKS proxies for S3 repository (opensearch-project#2160) Signed-off-by: Andrey Pleskach <ples@aiven.io> commit 3d5aff4 Author: Suraj Singh <79435743+dreamer-89@users.noreply.github.com> Date: Wed Mar 2 13:21:00 2022 -0800 Remove type end-points from search and related APIs (opensearch-project#2263) Signed-off-by: Suraj Singh <surajrider@gmail.com> commit 897f4e7 Author: Nick Knize <nknize@apache.org> Date: Wed Mar 2 13:44:04 2022 -0600 [Remove] deprecated getMapping API from IndicesClient (opensearch-project#2262) Removes the deprecated types based get, getMapping, getAsync, and getMappingAsync methods from IndicesClient. It also removes extra nesting of mappings belong the deprecated type named object and removes the types based methods from the affected request classes. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> commit 4b89410 Author: Breno Faria <30877433+br3no@users.noreply.github.com> Date: Wed Mar 2 19:52:38 2022 +0100 Reintroduce negative epoch_millis opensearch-project#1991 (opensearch-project#2232) * Reintroduce negative epoch_millis opensearch-project#1991 Fixes a regression introduced with Elasticsearch 7 regarding the date field type that removed support for negative timestamps with sub-second granularity. Thanks to Ryan Kophs (https://github.com/rkophs) for allowing me to use his previous work. Signed-off-by: Breno Faria <breno.faria@intrafind.de> * applying spotless fix Signed-off-by: Breno Faria <breno.faria@intrafind.de> * more conservative implementation of isSupportedBy Signed-off-by: Breno Faria <breno.faria@intrafind.de> * adding braces to control flow statement Signed-off-by: Breno Faria <breno.faria@intrafind.de> * spotless fix... Signed-off-by: Breno Faria <breno.faria@intrafind.de> Co-authored-by: Breno Faria <breno.faria@intrafind.de> commit 9e225dc Author: Peng Huo <penghuo@gmail.com> Date: Wed Mar 2 10:34:02 2022 -0800 Fix flaky test case - string profiler via global ordinals (opensearch-project#2226) forcemerge to one segment before executing aggregation query. Signed-off-by: Peng Huo <penghuo@gmail.com> commit c8a7606 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 12:18:28 2022 -0800 Bump commons-math3 from 3.2 to 3.6.1 in /benchmarks (opensearch-project#2282) Bumps commons-math3 from 3.2 to 3.6.1. --- updated-dependencies: - dependency-name: org.apache.commons:commons-math3 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 1b8181c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 12:17:16 2022 -0800 Bump gson from 2.8.9 to 2.9.0 in /plugins/repository-hdfs (opensearch-project#2279) * Bump gson from 2.8.9 to 2.9.0 in /plugins/repository-hdfs Bumps [gson](https://github.com/google/gson) from 2.8.9 to 2.9.0. - [Release notes](https://github.com/google/gson/releases) - [Changelog](https://github.com/google/gson/blob/master/CHANGELOG.md) - [Commits](google/gson@gson-parent-2.8.9...gson-parent-2.9.0) --- updated-dependencies: - dependency-name: com.google.code.gson:gson dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit 0df9845 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 12:16:56 2022 -0800 Bump morfologik-fsa from 2.1.1 to 2.1.8 in /plugins/analysis-ukrainian (opensearch-project#2278) * Bump morfologik-fsa from 2.1.1 to 2.1.8 in /plugins/analysis-ukrainian Bumps [morfologik-fsa](https://github.com/morfologik/morfologik-stemming) from 2.1.1 to 2.1.8. - [Release notes](https://github.com/morfologik/morfologik-stemming/releases) - [Changelog](https://github.com/morfologik/morfologik-stemming/blob/master/CHANGES.txt) - [Commits](morfologik/morfologik-stemming@2.1.1...2.1.8) --- updated-dependencies: - dependency-name: org.carrot2:morfologik-fsa dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit 9780fc6 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 12:16:34 2022 -0800 Bump bc-fips from 1.0.2.1 to 1.0.2.3 in /distribution/tools/plugin-cli (opensearch-project#2276) * Bump bc-fips from 1.0.2.1 to 1.0.2.3 in /distribution/tools/plugin-cli Bumps bc-fips from 1.0.2.1 to 1.0.2.3. --- updated-dependencies: - dependency-name: org.bouncycastle:bc-fips dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit 4ef30f4 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 10:41:46 2022 -0800 Bump azure-storage-common from 12.14.0 to 12.14.3 in /plugins/repository-azure (opensearch-project#2274) * Bump azure-storage-common in /plugins/repository-azure Bumps [azure-storage-common](https://github.com/Azure/azure-sdk-for-java) from 12.14.0 to 12.14.3. - [Release notes](https://github.com/Azure/azure-sdk-for-java/releases) - [Commits](Azure/azure-sdk-for-java@azure-storage-blob_12.14.0...azure-storage-blob_12.14.3) --- updated-dependencies: - dependency-name: com.azure:azure-storage-common dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit 5d0b015 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 10:41:33 2022 -0800 Bump jimfs from 1.1 to 1.2 in /distribution/tools/keystore-cli (opensearch-project#2272) Bumps [jimfs](https://github.com/google/jimfs) from 1.1 to 1.2. - [Release notes](https://github.com/google/jimfs/releases) - [Commits](google/jimfs@v1.1...v1.2) --- updated-dependencies: - dependency-name: com.google.jimfs:jimfs dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit f6264a9 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 10:41:18 2022 -0800 Bump spock-core from 2.0-groovy-3.0 to 2.1-groovy-3.0 in /buildSrc (opensearch-project#2270) Bumps spock-core from 2.0-groovy-3.0 to 2.1-groovy-3.0. --- updated-dependencies: - dependency-name: org.spockframework:spock-core dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> commit 21f11ec Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon Feb 28 10:40:51 2022 -0800 Bump asm-tree from 5.0.4 to 9.2 in /modules/lang-expression (opensearch-project#2269) * Bump asm-tree from 5.0.4 to 9.2 in /modules/lang-expression Bumps asm-tree from 5.0.4 to 9.2. --- updated-dependencies: - dependency-name: org.ow2.asm:asm-tree dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> * Updating SHAs Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com> commit 223d0b2 Author: Suraj Singh <79435743+dreamer-89@users.noreply.github.com> Date: Fri Feb 25 13:00:25 2022 -0800 Remove type end-points from no-op bulk and search action (opensearch-project#2261) Signed-off-by: Suraj Singh <surajrider@gmail.com> commit 0bd7850 Author: Nick Knize <nknize@apache.org> Date: Fri Feb 25 13:35:48 2022 -0600 [Remove] remaining type usage in Client and AbstractClient (opensearch-project#2258) Removes type parameter from remaining prepareIndex in Client and AbstractClient. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> commit 3a4c2f6 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri Feb 25 10:40:11 2022 -0800 Bump jimfs from 1.1 to 1.2 in /qa/evil-tests (opensearch-project#2130) * Bump jimfs from 1.1 to 1.2 in /qa/evil-tests Bumps [jimfs](https://github.com/google/jimfs) from 1.1 to 1.2. - [Release notes](https://github.com/google/jimfs/releases) - [Commits](google/jimfs@v1.1...v1.2) --- updated-dependencies: - dependency-name: com.google.jimfs:jimfs dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Fixing failing precommit and check Signed-off-by: Vacha Shah <vachshah@amazon.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Vacha Shah <vachshah@amazon.com> commit 5b0da85 Author: Suraj Singh <79435743+dreamer-89@users.noreply.github.com> Date: Fri Feb 25 10:18:41 2022 -0800 Remove type from validate query API (opensearch-project#2255) * Remove type mapping from RestValidateAction Signed-off-by: Suraj Singh <surajrider@gmail.com> * Spotless check apply Signed-off-by: Suraj Singh <surajrider@gmail.com> * Include suggested review comment Signed-off-by: Suraj Singh <surajrider@gmail.com> commit 494c7bc Author: Rishikesh Pasham <62345295+Rishikesh1159@users.noreply.github.com> Date: Fri Feb 25 03:39:43 2022 +0000 Revert "Override Default Distribution Download Url with Custom Distribution Url When User Passes a Url" (opensearch-project#2256) * Override default Distribution Download URL with custom Distribution URL Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Accidently made commit to main branch, this revives it.Override default Distribution Download URL with custom Distribution URL Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> * Revert Override Default Distribution Download Url with Custom Distribution Url When User Passes a Url Signed-off-by: Rishikesh1159 <rishireddy1159@gmail.com> commit 8b48207 Author: Nick Knize <nknize@apache.org> Date: Thu Feb 24 21:20:03 2022 -0600 [Remove] Type from Client.prepare(Index,Delete,Update) (opensearch-project#2253) Removes the type parameter from Client.prepare(Index,Delete,Update) and everywhere it's used throughout the codebase except for prepareIndex(index, type, id) which is removed in a follow up. Signed-off-by: Nicholas Walter Knize <nknize@apache.org> Merge branch 'main' into feature/segment-replication Signed-off-by: Kartik Ganesh <gkart@amazon.com>
Description
reindex.remote.allowlist
, aims to replace the existingreindex.remote.whitelist
reindex.remote.whitelist
as the fallback to the new settingreindex.remote.whitelist
This is the next step of the PR #2178
Expected behavior:
Deprecation notice:
With the built-in deprecation logger of OpenSearch (code), the following message will be recorded in the log files:
In logs/opensearch_deprecation.log:
[2022-03-07T20:48:57,158][DEPRECATION][o.o.d.c.s.Settings ] [ip-100-00-00-000] [reindex.remote.whitelist] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version.
In logs/opensearch_deprecation.json:
{"type": "deprecation", "timestamp": "2022-03-07T20:48:57,158Z", "level": "DEPRECATION", "component": "o.o.d.c.s.Settings", "cluster.name": "opensearch", "node.name": "ip-100-00-00-000", "message": "[reindex.remote.whitelist] setting was deprecated in OpenSearch and will be removed in a future release! See the breaking changes documentation for the next major version." }
Issues Resolved
Resolve #1547
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.