Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yaml modules for working towards troubleshooting easement #61183

Merged
merged 29 commits into from
Apr 6, 2022
Merged

yaml modules for working towards troubleshooting easement #61183

merged 29 commits into from
Apr 6, 2022

Conversation

whytewolf
Copy link
Collaborator

What does this PR do?

Adds a yaml module and yamllint utils module to work towards adding tools to better help in troubleshooting salt.

What issues does this PR fix or reference?

#61182

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

Commits signed with GPG?

No

Thomas Phipps added 4 commits November 2, 2021 21:23
eventually be yaml tools to help in troubleshooting yaml
functionality.
from __salt__.module.function to __salt__["module.function"] for the
test to work
@whytewolf whytewolf requested a review from a team as a code owner November 3, 2021 21:23
@whytewolf whytewolf requested review from joechainz and removed request for a team November 3, 2021 21:23
@github-actions
Copy link

github-actions bot commented Nov 5, 2021

Hi! I'm your friendly PR bot!

You might be wondering what I'm doing commenting here on your PR.

Yes, as a matter of fact, I am...

I'm just here to help us improve the documentation. I can't respond to
questions or anything, but what I can do, I do well!

Okay... so what do you do?

I detect modules that are missing docstrings or "CLI Example" on existing docstrings!
When I was created we had a lot of these. The documentation for these
modules need some love and attention to make Salt better for our users.

So what does that have to do with my PR?

I noticed that in this PR there are some files changed that have some of these
issues. So I'm leaving this comment to let you know your options.

Okay, what are they?

Well, my favorite, is that since you were making changes here I'm hoping that
you would be the most familiar with this module and be able to add some other
examples or fix any of the reported issues.

If I can, then what?

Well, you can either add them to this PR or add them to another PR. Either way is fine!

Well... what if I can't, or don't want to?

That's also fine! We appreciate all contributions to the Salt Project. If you
can't add those other examples, either because you're too busy, or unfamiliar,
or you just aren't interested, we still appreciate the contributions that
you've made already.

Whatever approach you decide to take, just drop a comment here letting us know!

Detected Issues (click me)
[INFO] Initializing environment for https://github.com/saltstack/pip-tools-compile-impersonate.
[INFO] Initializing environment for https://github.com/asottile/pyupgrade.
[INFO] Initializing environment for https://github.com/saltstack/pre-commit-remove-import-headers.
[INFO] Initializing environment for https://github.com/s0undt3ch/salt-rewrite.
[INFO] Initializing environment for https://github.com/timothycrosley/isort.
[INFO] Initializing environment for https://github.com/timothycrosley/isort:toml.
[INFO] Initializing environment for https://github.com/psf/black.
[INFO] Initializing environment for https://github.com/asottile/blacken-docs.
[INFO] Initializing environment for https://github.com/asottile/blacken-docs:black==21.7b0.
[INFO] Initializing environment for https://github.com/PyCQA/bandit.
[INFO] Initializing environment for https://github.com/saltstack/invoke-pre-commit.
[INFO] Initializing environment for https://github.com/saltstack/invoke-pre-commit:blessings,distro,jinja2,msgpack,pyyaml.
[INFO] Initializing environment for https://github.com/saltstack/mirrors-nox.
[INFO] Initializing environment for https://github.com/saltstack/mirrors-nox:pip>=20.2.4,<21.2,setuptools<58.0.
[INFO] Installing environment for https://github.com/saltstack/invoke-pre-commit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Check Known Missing Docstrings.......................(no files to check)Skipped
- hook id: invoke

Thanks again!

salt/modules/yaml.py Show resolved Hide resolved
salt/modules/yaml.py Show resolved Hide resolved
@whytewolf whytewolf requested a review from Ch3LL February 24, 2022 23:47
@Ch3LL Ch3LL added the Phosphorus v3005.0 Release code name and version label Mar 3, 2022
@whytewolf
Copy link
Collaborator Author

rerun all

@Ch3LL
Copy link
Contributor

Ch3LL commented Apr 6, 2022

yay finally!!!

@Ch3LL Ch3LL merged commit 062637a into saltstack:master Apr 6, 2022
@max-arnold
Copy link
Contributor

@whytewolf Hey! Trying the new yaml.lint function and it breaks in an obscure way. Ubuntu 20.04, apt install yamllint:

# phosphorus/foo.sls

foo:
  test.configurable_test_state:
    - changes: True
    - comment: A comment
salt-call yaml.lint salt://phosphorus/foo.sls

[ERROR   ] An un-handled exception was caught by Salt's global exception handler:
YamlLintConfigError: invalid config: unknown option "check-keys" for rule "truthy"
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    load_entry_point('salt==3003rc1+2349.g87f6189e58', 'console_scripts', 'salt-call')()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/scripts.py", line 441, in salt_call
    client.run()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/cli/call.py", line 50, in run
    caller.run()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/cli/caller.py", line 95, in run
    ret = self.call()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/cli/caller.py", line 202, in call
    ret["return"] = self.minion.executors[fname](
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/executors/direct_call.py", line 10, in execute
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/modules/yaml.py", line 56, in lint
    return __utils__["yamllint.lint"](yaml_out)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/utils/yamllint.py", line 49, in lint
    conf = YamlLintConfig(yamlconf)
  File "/usr/lib/python3/dist-packages/yamllint/config.py", line 43, in __init__
    self.validate()
  File "/usr/lib/python3/dist-packages/yamllint/config.py", line 121, in validate
    self.rules[id] = validate_rule_conf(rule, self.rules[id])
  File "/usr/lib/python3/dist-packages/yamllint/config.py", line 149, in validate_rule_conf
    raise YamlLintConfigError(
yamllint.config.YamlLintConfigError: invalid config: unknown option "check-keys" for rule "truthy"
Traceback (most recent call last):
  File "/usr/bin/salt-call", line 11, in <module>
    load_entry_point('salt==3003rc1+2349.g87f6189e58', 'console_scripts', 'salt-call')()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/scripts.py", line 441, in salt_call
    client.run()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/cli/call.py", line 50, in run
    caller.run()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/cli/caller.py", line 95, in run
    ret = self.call()
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/cli/caller.py", line 202, in call
    ret["return"] = self.minion.executors[fname](
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/executors/direct_call.py", line 10, in execute
    return func(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/modules/yaml.py", line 56, in lint
    return __utils__["yamllint.lint"](yaml_out)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 149, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1201, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/loader/lazy.py", line 1216, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt-3003rc1+2349.g87f6189e58-py3.8.egg/salt/utils/yamllint.py", line 49, in lint
    conf = YamlLintConfig(yamlconf)
  File "/usr/lib/python3/dist-packages/yamllint/config.py", line 43, in __init__
    self.validate()
  File "/usr/lib/python3/dist-packages/yamllint/config.py", line 121, in validate
    self.rules[id] = validate_rule_conf(rule, self.rules[id])
  File "/usr/lib/python3/dist-packages/yamllint/config.py", line 149, in validate_rule_conf
    raise YamlLintConfigError(
yamllint.config.YamlLintConfigError: invalid config: unknown option "check-keys" for rule "truthy"

Am I doing something wrong?

@whytewolf
Copy link
Collaborator Author

@max-arnold must be a version issue. what version of yamllint did you end up with that doesn't have check-keys as one of the truthy rules?

for the record here is the config used to get as close to salt style yaml as possible

        extends: relaxed
        rules:
          line-length: { max: 256, level: warning }
          empty-lines: disable
          empty-values: {forbid-in-block-mappings: false, forbid-in-flow-mappings: true}
          trailing-spaces: disable
          key-ordering: disable
          truthy: {level: warning, check-keys: false }

@max-arnold
Copy link
Contributor

The yamllint version is the default one on Ubuntu 20.04: 1.20.0-1

@max-arnold
Copy link
Contributor

Maybe a version-dependent config should be implemented? Or if that is too much hassle, then a minimal version check at the module's __init__()?

@whytewolf
Copy link
Collaborator Author

@max-arnold Can you create a new issue for this? I"ll try and get around to it soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Phosphorus v3005.0 Release code name and version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants