(Backport 50125, 50671, 51973, 52412, 52744, 52844, 54024, 54379) Add root and no_recommends parameters in the public interface for Zypper and RPM (lowpkg)

[aplanas]

What does this PR do?

(From #50125)
Provides a new root parameter in the public interfaces for rpm and zypper

(From #50671)
In _get_configured_repos if the directory where the repositories
are stored is not found, an log error is emitted.

Most of the times this is indeed an indication of an error, but is
not always the case. For example, when we are creating a chroot
environment we do not need to create this directory, as is generated
by zypper when a new repository is added.

This patch demote the error to a warning, as this directory will
be present eventually is most situations after the first call to
zypper.

(From #51973)
The virtual module 'pkg' is heavily used by other states, like the
'pkg' state, so differences in the signature of public functions
can generate errors like:

TypeError: xxx() got an unexpected keyword argument 'yyy'

The fix is to add the parameter 'yyy' into all versions of the
function 'xxx()' in all the 'pkg' virtual modules, but this
approach is cumbersome and not very resilient, as any refactoring
on the public interface or in the pkg state module will break
again the already agreed contract.

This patch append **kwarg in most of the public methods on virtual
module pkg where it is missing, fixing the described issue and
making more easy the refactoring of the code.

Also do the same for 'lowpkg'

(From #52412)

In the openSUSE family we have several kind of packages, like
patterns, patches and products. Currently we can use the pkg
state and the pkg module to install them, but as they are not
listed under any rpm command, the module and the state will
report a Fail.

This patch updates the zypperpkg list_pkgs to explicitly include
this kind of objects under the list, so the rest of the commands
from the same execution module (install, update, etc) will report
the correct restul at the end.

Because the way that the installed state from pkg is designed,
we cannot be implicit about the kind of objects that we are
installing, so we need to indicate in the SLS that at least one
object is a pattern or a patch:

  example:
    pkg.installed:
      - name: pattern:SUSE-MicroOS
      - includes: [pattern]

The includes parameter will be passed to the list_pkgs from
the zypper execution module, that will return the package list
using RPM (for packages) and zypper (for patterns). This more
complete report will be used in pkg.installed to identify the
correct installation of the new object.

Closes #52402

(From #52744)

Some patters are non-visible, and cannot be detected via

zypper se -t pattern

unless we know the name.

When we are listing all installed patterns, we still do not know
the names of the patterns, so we need to use a different approach.

This patch use rpm -q --provides --whatprovides search to detect
the non-visible patters, and complete the result for
list_installed_patterns.

(From #52844)

The `rpm -q --provides --whatprovides 'pattern()' command return
error code 1 when any pattern is present. This patch ignore the
return error code, as a not found pattern is not an error.

(From #54024)

The cache from pkg.list_pkgs for the zypper installer is too aggresive.
Some parameters will deliver different package lists, like root and
includes. The current cache do not take those parameters into
consideration, so the next time that this function is called, the last
list of packages will be returned, without checking if the current
parameters match the old one.

This patch create a different cache key for each parameter combination,
so the cached data will be separated too.

(From #54379)

When a patch is installed via advisory_ids, we do not want to change the
behavior of list_pkgs, via the includes field.

Currently, if a patch is installed via advisory_ids (state
pkg.patch_installed), it will call zypperpkg.install with this parameter
inside kwargs. This parameter is parsed by pkg_resource.parse_targets,
and the prefix patch: will be added very early, just before taking the
list of the current packages.

This last step will change the behavior of list_pkgs, as all the other
patches will be listed.

This sound OK except when we want to maintain the old behavior, when
patches are not listed in the install function by default.

This patch will add the patch: prefix in the list of targets later,
once we have the list of packages, restoring the old behavior.

Note that we can still install a patch as a normal package, if we add
the prefix in the name before calling install, and we do not use the
advisory_ids field.

What issues does this PR fix or reference?

New feature

Tests written?

Yes

(backport #50125, #50671, #51973, #52412, #52744, #52844 and #54024, already merged in develop)

(backport #54379 from develop, under review)

[aplanas]

Any change for this to be reviewed soon? It is actually a contention point, as this is a chain of changes that cover more than one year of commits, and there are more on the queue.

[aplanas]

Ping: If I split it into each backport, will help the review? The chain is very deep, and the all the different splits will contain all the previous PR, and the last one will contain all the PR like this one, so I am not sure if this will help or damage.

[aplanas]

no tests are failing, any re-review?