Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix opensearch-env always sources the environment from hardcoded file #875

Merged
merged 1 commit into from
Jul 21, 2021

Conversation

xuezhou25
Copy link
Contributor

@xuezhou25 xuezhou25 commented Jun 24, 2021

Description

Fix line 81 of opensearch-env always sources the environment from hardcoded file. Let users override the value of OPENSEARCH_PATH_CONF.

Issues Resolved

#633

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@opensearch-ci-bot
Copy link
Collaborator

❌   DCO Check Failed 3d313af3446fa7a7c5a1c9462e672101ca543673
Run ./dev-tools/signoff-check.sh remotes/origin/main 3d313af3446fa7a7c5a1c9462e672101ca543673 to check locally
Use git commit with -s to add 'Signed-of-by: {EMAIL}' on impacted commits

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success 3d313af3446fa7a7c5a1c9462e672101ca543673

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success 3d313af3446fa7a7c5a1c9462e672101ca543673

@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed c2c93ba

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success c2c93ba

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success c2c93ba

@nknize nknize added feedback needed Issue or PR needs feedback untriaged labels Jun 25, 2021
@@ -78,7 +78,9 @@ fi

export HOSTNAME=$HOSTNAME

${source.path.env}
if [ -z "$OPENSEARCH_PATH_CONF" ]; then
${source.path.env}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently this sources this file. This has other environment variables which won't be set if there is an environment variable OPENSEARCH_PATH_CONF in the current shell.
This might create problems without the user realizing why.

I think this needs be properly communicated in documentation and by logging a warning message.

Copy link
Collaborator

@tlfeng tlfeng Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree there needs a properly notice to the users to aware the other environment variable in ${source.path.env}, because there is one:

OPENSEARCH_STARTUP_SLEEP_TIME=5

But during my test, I found the variables in above file can still be set through systemd service file: . After the users commenting out this line manually, they can override the value of OPENSEARCH_PATH_CONF, and start several OpenSearch processes in the same host.
So in my opinion, the users understand what they are doing because they still need to do the above additional step, and common users will not be affected by the change in the PR.
While, if the user starts multiple OpenSearch processes without using the systemd service (though I don't know if there is a way to skip the systemd sevice), the value of OPENSEARCH_STARTUP_SLEEP_TIME is not set and the user may not realize.

Copy link
Contributor

@adnapibar adnapibar Jul 13, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's (OPENSEARCH_STARTUP_SLEEP_TIME=5 ) the only uncommented one but there are other variables that might have been modified. The service can be started without systemd. We can assume that the users know what they are doing but if we're changing existing behaviour it should be properly communicated.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Rabi 😄 I see, you are right.
With the change in the PR, when user set custom OPENSEARCH_PATH_CONF, and started OpenSearch process without systemd (such as running the opensearch bash script directly) the default values in ${source.path.env} will not be loaded, so it needs a notice.

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success c2c93ba

@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed c2c93ba

@nknize nknize added bug Something isn't working v1.1.0 Issues, PRs, related to the 1.1.0 release v2.0.0 Version 2.0.0 and removed feedback needed Issue or PR needs feedback labels Jul 2, 2021
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success c2c93ba

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Precommit failure 745155185fab8405353da16a9e3924c91bf8eb6a
Log 753

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success 745155185fab8405353da16a9e3924c91bf8eb6a

@opensearch-ci-bot
Copy link
Collaborator

❌   DCO Check Failed 745155185fab8405353da16a9e3924c91bf8eb6a
Run ./dev-tools/signoff-check.sh remotes/origin/main 745155185fab8405353da16a9e3924c91bf8eb6a to check locally
Use git commit with -s to add 'Signed-of-by: {EMAIL}' on impacted commits

@opensearch-ci-bot
Copy link
Collaborator

❌   DCO Check Failed 3f259df25a79e5e3031efb401ca0fa2d70563375
Run ./dev-tools/signoff-check.sh remotes/origin/main 3f259df25a79e5e3031efb401ca0fa2d70563375 to check locally
Use git commit with -s to add 'Signed-of-by: {EMAIL}' on impacted commits

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success 3f259df25a79e5e3031efb401ca0fa2d70563375

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success 3f259df25a79e5e3031efb401ca0fa2d70563375

@xuezhou25 xuezhou25 force-pushed the new_branch branch 2 times, most recently from 7451551 to c2c93ba Compare July 7, 2021 06:43
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success 745155185fab8405353da16a9e3924c91bf8eb6a

@opensearch-ci-bot
Copy link
Collaborator

✅   DCO Check Passed c2c93ba

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Precommit failure 745155185fab8405353da16a9e3924c91bf8eb6a
Log 755

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Wrapper Validation success c2c93ba

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Precommit success c2c93ba

@dblock
Copy link
Member

dblock commented Jul 12, 2021

@xuezhou25 what is the original problem that you are trying to fix with this? why would you need this override?

@adnapibar It sounds like we don't want this change because of the undesirable side effects?

@xuezhou25
Copy link
Contributor Author

@xuezhou25 what is the original problem that you are trying to fix with this? why would you need this override?

The original problem is https://github.com/opensearch-project/OpenSearch/issues/633
When users want to start multiple OpenSearch instances on the same host, with deb or rpm package, they need to override the default configuration path $OPENSEARCH_PATH_CONF with multiple different values for multiple instances to use. But with the line of code in opensearch-env https://github.com/opensearch-project/OpenSearch/blob/34550c5b17124ddc59458ef774f6b43a086522e3/distribution/src/bin/opensearch-env#L81 the environment viable $OPENSEARCH_PATH_CONF can only be set to the single value in ${source.path.env}.
Shall we communicate this in documentation?

@dblock
Copy link
Member

dblock commented Jul 14, 2021

@xuezhou25 what is the original problem that you are trying to fix with this? why would you need this override?

The original problem is https://github.com/opensearch-project/OpenSearch/issues/633
When users want to start multiple OpenSearch instances on the same host, with deb or rpm package, they need to override the default configuration path $OPENSEARCH_PATH_CONF with multiple different values for multiple instances to use. But with the line of code in opensearch-env https://github.com/opensearch-project/OpenSearch/blob/34550c5b17124ddc59458ef774f6b43a086522e3/distribution/src/bin/opensearch-env#L81 the environment viable $OPENSEARCH_PATH_CONF can only be set to the single value in ${source.path.env}.
Shall we communicate this in documentation?

Ok, that is clear. So this is a feature that allows one to override OPENSEARCH_PATH_CONF. We definitely need to at least document this. We should also be testing starting multiple instances with deb/rpm packages, so:

  1. Open an issue in the doc repo to document this.
  2. Open an issue here to test this unless the PR contains such tests.

With ^ this LGTM.

@dblock
Copy link
Member

dblock commented Jul 14, 2021

start gradle check

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success c2c93ba
Log 334

Reports 334

@adnapibar
Copy link
Contributor

@adnapibar It sounds like we don't want this change because of the undesirable side effects?

There might be some edge cases and you're right we do need some tests to verify. But at least documenting it should make it clear to the users.

@adnapibar adnapibar merged commit e3d86ba into opensearch-project:main Jul 21, 2021
peterzhuamazon pushed a commit to peterzhuamazon/OpenSearch that referenced this pull request Apr 14, 2022
…opensearch-project#875)

distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: Xue Zhou <xuezhou@amazon.com>
peterzhuamazon pushed a commit to peterzhuamazon/OpenSearch that referenced this pull request Apr 14, 2022
…opensearch-project#875)

distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: xuezhou25 <85715413+xuezhou25@users.noreply.github.com>
Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
peterzhuamazon pushed a commit to peterzhuamazon/OpenSearch that referenced this pull request Apr 14, 2022
…opensearch-project#875)

distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: xuezhou25 <85715413+xuezhou25@users.noreply.github.com>
Signed-off-by: Peter Zhu <zhujiaxi@amazon.com>
tlfeng pushed a commit that referenced this pull request Apr 14, 2022
…#875) (#2908)

backport commit e3d86ba to 1.x branch

distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: xuezhou25 <xuezhou25@amazon.com>
tlfeng pushed a commit that referenced this pull request Apr 14, 2022
…#875) (#2909)

backport commit e3d86ba to 1.3 branch

distribution/bin/opensearch-env always sources the environment from the default environment file /etc/default/opensearch. This is an issue if we want to run multiple instances of OpenSearch on the same host. This change lets users override the default behavior by not sourcing the default environment file in case OPENSEARCH_PATH_CONF  is set.

Signed-off-by: xuezhou25 <xuezhou25@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v1.1.0 Issues, PRs, related to the 1.1.0 release v2.0.0 Version 2.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants