Package environment

[isbm]

What does this PR do?

• Adds feature of configurable environment for the modules
• Numerous bugfixes for Apt and Yum/Dnf

New Behavior

It is now possible to configure an environment variables for each module and even its functions separately and reuse it with the cmd.run* utility (or other similar matters). This is especially useful if there are package manager plugins that are communicating via environment variables. You find that a lot in Apt/dpkg module. E.g.: write a plugin for a package manager that detects (and records) an event so the package has been updated/installed just directly manually or from within Salt.

Tests written?

Yes

[meaksh]

Looks good to me! Thanks for improving this!

[terminalmage]

As noted in the header of this file, we are not adding new functions here. I don't see a natural home in the existing modules under salt/utils/, so this may need to go into its own file. salt/utils/environment.py perhaps?

[isbm]

@terminalmage who reads the headers... my bad. But yeah, I thought about exactly this but hesitated. OK, salt/utils/environment.py it is.

[terminalmage]

I'm not sure I like this structure. A lot of users aren't developers, and will thus not intrinsically grok the "module path" naming. From a user-friendliness standpoint, I think it would be more intuitive to do something like:

system-environment:
  modules:
    pkg:
      FOO: bar
  states:
    pkg:
      HELLO: world

@cachedout thoughts on this?

[isbm]

@cachedout please agree with the @terminalmage because I like his proposal more than mine. 😉

[isbm]

@terminalmage Fixed. With even different way:

system-environment:
  modules:
    pkg:
      _:
        FOR: all
      install:
        FOR: install-only
  states:
    pkg:
      _:
        HELLO: world

In some special case, when someone wants to make sure pkg.foo has different stuff than pkg.bar, say a Yum plugin wants to react differently if you installing or upgrading or updating; or Nagios wants to deploy a plugin or fetch it etc.

[isbm]

@rallytime ha, seems like not everything was mocked for Yum/Dnf unit tests! The systemd capability is actually called for real, hence on my side this test has been never failed, but does fail here. Will fix that.

[terminalmage]

@isbm thanks, will re-review on Monday

[isbm]

re-run py

[terminalmage]

We either need to use the full module namespace, or we need to import the function as a private function (i.e. as _get_module_environment). The reason for this is because the loader will add all public functions in the module into the __salt__ dunder.

[terminalmage]

This will force a user to set LC_ALL and LANG. Would it be a good idea to set up some sort of default environment? Since we're passing globals() to get_module_environment(), we could maybe define it as a dunder in the global scope?

[isbm]

These are just a left-overs from older times, since we're now got run_all properly and resetting the locale there right away.

[terminalmage]

It's rare that we need to do ignore the retcode, I don't think this should be the default. Why not just remove this line and pass through ignore_retcode=True in the kwargs in the two places where we use it (autoremove and list_repo_pkgs)?

[isbm]

Yes, that's would do, of course.

[terminalmage]

Same as with apt, this would add get_module_environment to the __salt__ dunder.

[terminalmage]

Same as with apt, I don't think we should make this the default. Turning this on means we won't get any logging when a command exits with a nonzero exit status. There are instances where this is useful, as some commands return nonzero exit status in non-error cases, but it's an exception, not the norm.

[isbm]

Zypper actually doesn't do this. But I just moved it as a default behaviour (most of the time) in yumpkg module. By not ignoring the return code, could it be that the Salt won't succeed anymore running Yum? 😆

[terminalmage]

Same as apt and yum, this will put get_module_environment in the __salt__ dunder.

[isbm]

@terminalmage done.