Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the Salt Proxy environment aware #56222

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

bdrung
Copy link
Contributor

@bdrung bdrung commented Feb 21, 2020

The Salt proxy minion is looking for proxy minion modules in salt://_proxy/. It does so however only in the default base environment. On setups which do not use base or shall be executed in a different environment this breaks:

/etc/salt/master:

...
file_roots:
  noc:
    - /srv/salt
...
$ grep "proxyenabled" /srv/salt/_proxy/junos_manager.py
__proxyenabled__ = ['junos_manager']

$ salt-proxy --proxyid=dev1 -l debug
...
[DEBUG   ] rest_sample proxy __virtual__() called...
[INFO    ] ssh_sample proxy __virtual__() called...
[DEBUG   ] Could not LazyLoad junos_manager.grains
[DEBUG   ] Could not LazyLoad junos_manager.init
[ERROR   ] Proxymodule junos_manager is missing an init() or a
shutdown() or both. Check your proxymodule.  Salt-proxy aborted.
[WARNING ] Stopping the Salt Proxy Minion
[ERROR   ] -1
[INFO    ] The proxy minion is shutting down..
[INFO    ] The Salt ProxyMinion is shut down

This is because the loader only looks for _proxy modules in the base environment. This commit fixes this (but might possibly break other things, though I did not find side-effects).

The initial pull request #36704 were merged quite some time ago, but one part of it is missing in the 2019.2 release (again?).

@bdrung bdrung requested a review from a team as a code owner Feb 21, 2020
@ghost ghost requested a review from twangboy Feb 21, 2020
twangboy
twangboy previously approved these changes Mar 5, 2020
@sagetherage sagetherage added this to In progress in Core Open PRs Mar 19, 2020
@sagetherage sagetherage added the ZRelease-Sodium retired label label Mar 23, 2020
@sagetherage sagetherage added this to In progress in Sodium via automation Mar 23, 2020
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch from a5556d4 to 4315cab Compare Apr 7, 2020
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch from a058a63 to afe5a78 Compare Apr 14, 2020
@twangboy
Copy link
Contributor

twangboy commented Apr 15, 2020

Hey, @bdrung

Looks like this needs to be rebased and have pre-commit run on it. Do you mind doing that?

https://docs.saltstack.com/en/latest/topics/development/contributing.html#quickstart

@bdrung
Copy link
Contributor Author

bdrung commented Apr 15, 2020

I rebased it yesterday and pre-commit was successfully run.

@twangboy
Copy link
Contributor

twangboy commented Apr 15, 2020

Sweet... now we need to figure out why these tests are failing

@bdrung
Copy link
Contributor Author

bdrung commented Apr 16, 2020

Only codecov/project is failing, which makes no sense since this merge request just changes one line.

@waynew
Copy link
Contributor

waynew commented Apr 16, 2020

codecov can be ignored - on the newest master it should be.

Question: Why is it "saltenv" on this branch, but "environment" on the develop branch PR?

@bdrung
Copy link
Contributor Author

bdrung commented Apr 17, 2020

In one salt release, the environment option was replaced by saltenv.

@dwoz dwoz added the Needs-Testcase PR needs test cases written, or the issue is about a bug/feature that needs test cases label Apr 19, 2020
waynew
waynew previously approved these changes Apr 21, 2020
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch 6 times, most recently from 132df01 to 1b5fbbd Compare Apr 30, 2020
twangboy
twangboy previously approved these changes May 21, 2020
@twangboy
Copy link
Contributor

twangboy commented May 21, 2020

@bdrung Unless this gets a test today, it's not going to make it into Sodium.

@dwoz dwoz added the help-wanted Community help is needed to resolve this label May 22, 2020
@sagetherage sagetherage added Magnesium Mg release after Na prior to Al and removed ZRelease-Sodium retired label labels May 26, 2020
@sagetherage sagetherage removed this from In progress in Sodium May 26, 2020
@sagetherage sagetherage removed the Magnesium Mg release after Na prior to Al label Jun 1, 2020
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch 2 times, most recently from 8d47f0c to 127ffac Compare Jul 31, 2020
@twangboy twangboy force-pushed the make-salt-proxy-environment-aware branch from 127ffac to f2ae215 Compare Sep 23, 2020
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch from f2ae215 to cf60860 Compare Oct 7, 2020
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch from cf60860 to 3ce4f41 Compare Jun 2, 2021
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch from 3ce4f41 to b402205 Compare Nov 11, 2021
@github-actions
Copy link

github-actions bot commented Nov 11, 2021

Hi! I'm your friendly PR bot!

You might be wondering what I'm doing commenting here on your PR.

Yes, as a matter of fact, I am...

I'm just here to help us improve the documentation. I can't respond to
questions or anything, but what I can do, I do well!

Okay... so what do you do?

I detect modules that are missing docstrings or "CLI Example" on existing docstrings!
When I was created we had a lot of these. The documentation for these
modules need some love and attention to make Salt better for our users.

So what does that have to do with my PR?

I noticed that in this PR there are some files changed that have some of these
issues. So I'm leaving this comment to let you know your options.

Okay, what are they?

Well, my favorite, is that since you were making changes here I'm hoping that
you would be the most familiar with this module and be able to add some other
examples or fix any of the reported issues.

If I can, then what?

Well, you can either add them to this PR or add them to another PR. Either way is fine!

Well... what if I can't, or don't want to?

That's also fine! We appreciate all contributions to the Salt Project. If you
can't add those other examples, either because you're too busy, or unfamiliar,
or you just aren't interested, we still appreciate the contributions that
you've made already.

Whatever approach you decide to take, just drop a comment here letting us know!

Detected Issues (click me)
[INFO] Initializing environment for https://github.com/saltstack/pip-tools-compile-impersonate.
[INFO] Initializing environment for https://github.com/asottile/pyupgrade.
[INFO] Initializing environment for https://github.com/saltstack/pre-commit-remove-import-headers.
[INFO] Initializing environment for https://github.com/s0undt3ch/salt-rewrite.
[INFO] Initializing environment for https://github.com/timothycrosley/isort.
[INFO] Initializing environment for https://github.com/timothycrosley/isort:toml.
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Initializing environment for https://github.com/asottile/blacken-docs.
[INFO] Initializing environment for https://github.com/asottile/blacken-docs:black==21.7b0.
[INFO] Initializing environment for https://github.com/PyCQA/bandit.
[INFO] Initializing environment for https://github.com/saltstack/invoke-pre-commit.
[INFO] Initializing environment for https://github.com/saltstack/invoke-pre-commit:blessings,distro,jinja2,msgpack,pyyaml.
[INFO] Initializing environment for https://github.com/saltstack/mirrors-nox.
[INFO] Initializing environment for https://github.com/saltstack/mirrors-nox:pip>=20.2.4,<21.2,setuptools<58.0.
[INFO] Installing environment for https://github.com/saltstack/invoke-pre-commit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Check Known Missing Docstrings...........................................Passed
- hook id: invoke
- duration: 1.95s

Thanks again!

The Salt proxy minion is looking for proxy minion modules in
`salt://_proxy/`. It does so however only in the default `base`
environment. On setups which do not use `base` or shall be executed in a
different environment this breaks:

/etc/salt/master:
```
...
file_roots:
  noc:
    - /srv/salt
...
```

```
$ grep "proxyenabled" /srv/salt/_proxy/junos_manager.py
__proxyenabled__ = ['junos_manager']

$ salt-proxy --proxyid=dev1 -l debug
...
[DEBUG   ] rest_sample proxy __virtual__() called...
[INFO    ] ssh_sample proxy __virtual__() called...
[DEBUG   ] Could not LazyLoad junos_manager.grains
[DEBUG   ] Could not LazyLoad junos_manager.init
[ERROR   ] Proxymodule junos_manager is missing an init() or a
shutdown() or both. Check your proxymodule.  Salt-proxy aborted.
[WARNING ] Stopping the Salt Proxy Minion
[ERROR   ] -1
[INFO    ] The proxy minion is shutting down..
[INFO    ] The Salt ProxyMinion is shut down
```

This is because the loader only looks for _proxy modules in the `base`
environment. This commit fixes this (but might possibly break other
things, though I did not find side-effects).

Initial pull request: saltstack#36704
Forwarded: saltstack#55932
Signed-off-by: Benjamin Drung <benjamin.drung@cloud.ionos.com>
@bdrung bdrung force-pushed the make-salt-proxy-environment-aware branch from b402205 to e8a75e5 Compare Dec 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has-failing-test help-wanted Community help is needed to resolve this Needs-Testcase PR needs test cases written, or the issue is about a bug/feature that needs test cases
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants