pkg5 fails if asked to install a package that is already present

[autarch]

ISSUE TYPE

• Bug Report

COMPONENT NAME

• pkg5 module

ANSIBLE VERSION

ansible 2.2.1.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

CONFIGURATION

No changes to config file and no env vars

OS / ENVIRONMENT

This is specific to a Solaris host. In this particular case I'm using Solaris 11

SUMMARY

The short version is that pkg install will return 4 as its return code if the package already exists, and that needs to be treated as successful by the module.

This was already discussed a fair bit in ansible/ansible-modules-extras#932 and ansible/ansible-modules-extras#933

The weird thing is that this only seems to happen with a subset of packages.

STEPS TO REPRODUCE

Here's a task that causes this issue to happen for me.

---
- name: Make sure all packages we want are installed
  pkg5:
    name:
      - gcc
      - /editor/gnu-emacs/gnu-emacs-no-x11
      - vim
    state: latest

The package that causes the problem here is "vim".

EXPECTED RESULTS

I thought that it should just ignore packages that already installed.

ACTUAL RESULTS

ASK [solaris : Make sure all packages we want are installed] ******************
task path: /home/autarch/work/camel/ansible/roles/solaris/tasks/packages.yml:2
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/extras/packaging/os/pkg5.py
<192.168.42.108> ESTABLISH SSH CONNECTION FOR USER: ansible
<192.168.42.108> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ansible -o ConnectTimeout=10 -o ControlPath=/home/autarch/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.42.108 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo ~/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729 `" && echo ansible-tmp-1489996773.76-54219734952729="` echo ~/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729 `" ) && sleep 0'"'"''
<192.168.42.108> PUT /tmp/tmpclxp6n TO /export/home/ansible/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729/pkg5.py
<192.168.42.108> SSH: EXEC sftp -b - -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ansible -o ConnectTimeout=10 -o ControlPath=/home/autarch/.ansible/cp/ansible-ssh-%h-%p-%r '[192.168.42.108]'
<192.168.42.108> ESTABLISH SSH CONNECTION FOR USER: ansible
<192.168.42.108> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ansible -o ConnectTimeout=10 -o ControlPath=/home/autarch/.ansible/cp/ansible-ssh-%h-%p-%r 192.168.42.108 '/bin/sh -c '"'"'chmod u+x /export/home/ansible/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729/ /export/home/ansible/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729/pkg5.py && sleep 0'"'"''
<192.168.42.108> ESTABLISH SSH CONNECTION FOR USER: ansible
<192.168.42.108> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ansible -o ConnectTimeout=10 -o ControlPath=/home/autarch/.ansible/cp/ansible-ssh-%h-%p-%r -tt 192.168.42.108 '/bin/sh -c '"'"'sudo -H -S  -p "[sudo via ansible, key=kaascrakupualwmqivjlpjpcbinptsjl] password: " -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-kaascrakupualwmqivjlpjpcbinptsjl; /usr/bin/python /export/home/ansible/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729/pkg5.py; rm -rf "/export/home/ansible/.ansible/tmp/ansible-tmp-1489996773.76-54219734952729/" > /dev/null 2>&1'"'"'"'"'"'"'"'"' && sleep 0'"'"''
fatal: [192.168.42.108]: FAILED! => {
    "changed": true, 
    "failed": true, 
    "invocation": {
        "module_args": {
            "accept_licenses": false, 
            "name": [
                "gcc", 
                "/editor/gnu-emacs/gnu-emacs-no-x11", 
                "vim"
            ], 
            "state": "latest"
        }, 
        "module_name": "pkg5"
    }, 
    "msg": "", 
    "rc": 4, 
    "results": [
        ""
    ]
}

[ansibot]

cc @mavit
click here for bot help

[mavit]

What does pkg list -af \*vim\* say? How about pkg publisher?

needs_info

[autarch]

ansible@solaris:~$ pkg list -af \*vim\*
NAME (PUBLISHER)                                  VERSION                    IFO
SUNWgvim                                          7.2.308-0.133              --r
SUNWvim                                           7.2.308-0.133              --r
editor/gvim                                       8.0.95-5.12.0.0.0.115.0    ---
editor/gvim                                       7.4-5.12.0.0.0.105.1       ---
editor/gvim                                       7.4-5.12.0.0.0.95.0        ---
editor/gvim                                       7.4-5.12.0.0.0.90.0        ---
editor/gvim                                       7.3.600-0.175.3.0.0.30.0   ---
editor/gvim                                       7.3.600-0.175.3.0.0.26.0   ---
editor/gvim                                       7.3.600-0.175.2.0.0.42.1   ---
editor/gvim                                       7.3.600-0.175.1.0.0.24.0   ---
editor/gvim                                       7.3.421-0.175.0.10.1.0.0   ---
editor/gvim                                       7.3.254-0.175.0.0.0.2.537  ---
editor/gvim                                       7.2.308-0.151.0.1          ---
editor/vim                                        8.0.95-5.12.0.0.0.115.0    ---
editor/vim                                        7.4-5.12.0.0.0.105.1       ---
editor/vim                                        7.4-5.12.0.0.0.95.0        ---
editor/vim                                        7.4-5.12.0.0.0.90.0        ---
editor/vim                                        7.3.600-0.175.3.0.0.30.0   i--
editor/vim                                        7.3.600-0.175.3.0.0.26.0   ---
editor/vim                                        7.3.600-0.175.2.0.0.42.1   ---
editor/vim                                        7.3.600-0.175.1.0.0.24.0   ---
editor/vim                                        7.3.421-0.175.0.10.1.0.0   ---
editor/vim                                        7.3.254-0.175.0.0.0.2.537  ---
editor/vim                                        7.2.308-0.151.0.1          ---
editor/vim/vim-core                               8.0.95-5.12.0.0.0.115.0    ---
editor/vim/vim-core                               7.4-5.12.0.0.0.105.1       ---
editor/vim/vim-core                               7.4-5.12.0.0.0.95.0        ---
editor/vim/vim-core                               7.4-5.12.0.0.0.90.0        ---
editor/vim/vim-core                               7.3.600-0.175.3.0.0.30.0   i--
editor/vim/vim-core                               7.3.600-0.175.3.0.0.26.0   ---
editor/vim/vim-core                               7.3.600-0.175.2.0.0.42.1   ---
editor/vim/vim-core                               7.3.600-0.175.1.0.0.24.0   ---
editor/vim/vim-core                               7.3.421-0.175.0.10.1.0.0   ---
editor/vim/vim-core                               7.3.254-0.175.0.0.0.2.537  ---

[autarch]

Hmm, on further investigation, this seems to cause none of the packages specified in the list to be installed, so even if I add a failed_when to work around the rc 4, something else is going on with the pkg install command that is causing a problem.

[autarch]

I think this may be a bit of a red herring. On further investigation I realized that setting state: latest means that nothing gets installed at all! See #22823 for details.

[mavit]

You say you're running Solaris 11, but I notice that the newer vim packages are part of (the as yet unreleased) Solaris 12.

What does pkg list -u vim say? Does pkg change-facet version-lock.editor/vim=false make a difference?

I think what's happening here is that pkg is declining to upgrade you to Solaris 12 simply to pull in a newer version of vim. This seems like reasonable behaviour, but the documentation for the latest option could certainly stand to be better.

[mavit]

needs_info

[autarch]

ansible@solaris:~$ pkg list -u vim
NAME (PUBLISHER)                                  VERSION                    IFO
editor/vim                                        7.3.600-0.175.3.0.0.30.0   i--

Running pkg change-facet version-lock.editor/vim=false did not change the return code from pkg install vim, which was still 4.

[autarch]

I tried out the proposed fix for #22823 and that didn't change the issue I was seeing here, so I now think they're two completely separate issues.

I do note that if I ask for the vim package to be present instead of latest the issue goes away, because of course vim is already installed by default. That said, trying to upgrade a default package shouldn't be an error.

[mkrizek]

@autarch can you please confirm that #23007 fixes the issue? Thanks!

[danowar2k]

I did some tests and found the reason for the strange behaviour. It has to do with restricting versions in Solaris.

# Check installed wget version
admin@mahara-test:~$ pkg list web/wget
NAME (PUBLISHER)                                  VERSION                    IFO
web/wget                                          1.18-0.175.3.11.0.4.0      i--
admin@mahara-test:~$ echo $?
0
# Check if update possible
admin@mahara-test:~$ pkg list -a web/wget
NAME (PUBLISHER)                                  VERSION                    IFO
web/wget                                          1.18-0.175.3.11.0.4.0      i--
admin@mahara-test:~$ echo $?
0
# Update possible. Just try to update
admin@mahara-test:~$ sudo pkg update web/wget
Password:
No updates available for this image. (zone:mahara-test)
admin@mahara-test:~$ echo $?
4
# It seems I could update, but I can't because of some restriction.
# Is there really a new wget version available? Checking...
admin@mahara-test:~$ pkg list -af web/wget
NAME (PUBLISHER)                                  VERSION                    IFO
web/wget                                          1.19.2-0.175.3.27.0.2.0    ---
web/wget                                          1.18-0.175.3.11.0.4.0      i--
#[...]
admin@mahara-test:~$ echo $?
0
# Yes, there is a newer one, but it seems it is from the FOSS evaluation package, if I recall correctly.
# Now we try to explicitly request the latest package
admin@mahara-test:~$ sudo pkg update web/wget@latest
Creating Plan (Solver setup): /
pkg update: No matching version of web/wget can be installed:
  Reject:  pkg://solaris/web/wget@1.19.2-0.175.3.27.0.2.0
  Reason:  This version is excluded by installed incorporation consolidation/userland/userland-incorporation@0.5.11-0.175.3.25.0.3.0
admin@mahara-test:~$ echo $?
1

There's the reason we can't update to the latest package. Solaris has packages which freeze the system to some particular package level.
See https://docs.oracle.com/cd/E26502_01/html/E21383/glyus.html#scrolltoc (Incorporation packages)
The package userland-incorporation says that even though a new wget version exists, we can't install it right now.
This mechanism is to ensure that the toolsets work in an optimal way together.

Now we either ignore the rc 4 when we want the latest package or...add a switch that we try to update other packages accordingly?
Maybe there is an update to the userland incorporation package that allows the new wget version?

Or we allow restrictions to be relaxed with a switch?
(See https://docs.oracle.com/cd/E26502_01/html/E28984/gmias.html)

[danowar2k]

So... the latest userland-incorporation would allow the latest wget version.

But it can't be updated because it needs the latest "entire" incorporation package.

And that itself can't be updated because the global zone doesn't provide the newest version for it.

So in an effective global managed installation we are depending on packages "released" through our gloabl zones, because they are deemed "good". Well okay, but then we should maybe just allow rc 4 after all ;-)

RC 4 is saying "The latest version I can provide you with is already installed. Everything else is currently impossible."

[danowar2k]

Thus I can confirm that the patch for #23007 would fix the problem. Please @autarch confirm the fix if you have to so that the patch can be merged. If my word is enough, all the better ;-)

[autarch]

Unfortunately, this was a for work thing that is no longer being worked on, so I don't think I'm going to be able to check :(