user.py fails on generate_ssh_key under Python3.6

[gaddman]

ISSUE TYPE

• Bug Report

COMPONENT NAME

user.py

ANSIBLE VERSION

ansible 2.5.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/myuser/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

CONFIGURATION

DEFAULT_FORKS(/etc/ansible/ansible.cfg) = 100
DEFAULT_LOG_PATH(/etc/ansible/ansible.cfg) = /var/log/ansible.log
DEFAULT_SSH_TRANSFER_METHOD(/etc/ansible/ansible.cfg) = scp
DEFAULT_TIMEOUT(/etc/ansible/ansible.cfg) = 20
RETRY_FILES_ENABLED(/etc/ansible/ansible.cfg) = False

OS / ENVIRONMENT

Running from: Ubuntu 16.04.4 LTS
Managing: Ubuntu 18.04 LTS with Python 3.6

SUMMARY

User module fails when using Python3.6 and generate_ssh_key=yes.

STEPS TO REPRODUCE

With an SSH key already generated:

ansible myhost -vvv -m user -a "name=myuser generate_ssh_key=yes" -e ansible_python_interpreter=/usr/bin/python3

EXPECTED RESULTS

With Python2 (using -e ansible_python_interpreter=/usr/bin/python)

ok: [myhost] => {
    "append": false,
    "changed": false,
    "comment": "myuser,,,",
    "group": 1000,
    "home": "/home/myuser",
    "invocation": {
        "module_args": {
            "append": false,
            "comment": null,
            "create_home": true,
            "expires": null,
            "force": false,
            "generate_ssh_key": true,
            "group": null,
            "groups": null,
            "home": null,
            "local": null,
            "login_class": null,
            "move_home": false,
            "name": "myuser",
            "non_unique": false,
            "password": null,
            "remove": false,
            "seuser": null,
            "shell": null,
            "skeleton": null,
            "ssh_key_bits": 0,
            "ssh_key_comment": "ansible-generated on myhost",
            "ssh_key_file": null,
            "ssh_key_passphrase": null,
            "ssh_key_type": "rsa",
            "state": "present",
            "system": false,
            "uid": null,
            "update_password": "always"
        }
    },
    "move_home": false,
    "name": "myuser",
    "shell": "/bin/bash",
    "ssh_fingerprint": "2048 SHA256:Y4nt6r7XIwhLA3hHl1mxlJLfWKRqUY11yEzpHBO2Ruw ansible-generated on fender (RSA)",
    "ssh_key_file": "/home/myuser/.ssh/id_rsa",
    "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDfLscrODZb5hyh+ksFwv7zw2DR5SCxdjWcJC6zf/QO+3L+DS8wEZoQsQmP+5H1+MJ5SLMzfaV3VPOr3MJA7oARdOXhiEvRtj27kg8KmXJ5q6nYlM0lEiNIEaIQuyJxpQ7qKJwKbDb6fxlQnTX9tch/PD7xxrw2jCiibelHaTn5rg+180nTizhtdyNwp4nQ3U176b9qEm3KDjL9dUJQ6CTFBOK8nBVKfXan+OvlR0xgFlm9GeBTLGrz5uzJxEX769IE0I04gUlV5WBI6eCMqJzUm/uJQVB4AlSeRsWUpyImcgFYObdTMC1pKyjSIim5teXD9TWXnPiC+eNs7jdYzsmR ansible-generated on host",
    "state": "present",
    "uid": 1000

ACTUAL RESULTS

With Python3:

fatal: [myhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Shared connection to localhost closed.\r\n",
    "module_stdout": "Traceback (most recent call last):\r\n  File \"/tmp/ansible_xydrgos5/ansible_module_user.py\", line 2255, in <module>\r\n    main()\r\n  File \"/tmp/ansible_xydrgos5/ansible_module_user.py\", line 2193, in main\r\n    (rc, out, err) = user.modify_user()\r\n  File \"/tmp/ansible_xydrgos5/ansible_module_user.py\", line 698, in modify_user\r\n    return self.modify_user_usermod()\r\n  File \"/tmp/ansible_xydrgos5/ansible_module_user.py\", line 455, in modify_user_usermod\r\n    info = self.user_info()\r\n  File \"/tmp/ansible_xydrgos5/ansible_module_user.py\", line 599, in user_info\r\n    info[1] = self.user_password()\r\n  File \"/tmp/ansible_xydrgos5/ansible_module_user.py\", line 606, in user_password\r\n    passwd = spwd.getspnam(self.name)[1]\r\nPermissionError: [Errno 13] Permission denied\r\n",
    "msg": "MODULE FAILURE",
    "rc": 1

If the -b flag is used then it works successfully.

FURTHER TESTING

Looking into this a bit more, the behaviour of spwd.getspnam has changed in Python 3.6. From the docs: Raises a PermissionError instead of KeyError if the user doesn’t have privileges.

Python2 non-root:

myuser@myhost:~$ python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import spwd
>>> spwd.getspnam('myuser')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'getspnam(): name not found'
>>>

Python3 non-root:

myuser@myhost:~$ python3
Python 3.6.5 (default, Apr  1 2018, 05:46:30)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import spwd
>>> spwd.getspnam('myuser')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [Errno 13] Permission denied
>>>

[ansibot]

Files identified in the description:

• lib/ansible/modules/system/user.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibot]

cc @sfromm
click here for bot help

[gaddman]

I don't know if this is a suitable fix, but it did get me working again. Add the 2nd exception here after line 607:

            try:
                passwd = spwd.getspnam(self.name)[1]
            except KeyError:
                return passwd
            except OSError as e:
                if e.errno==13:
                    return passwd

Python didn't have the PermissionError until v3.3, so have to check for an OSError in Py2.7.

[gtema]

agree with gaddman. According to https://docs.python.org/3/library/spwd.html was changed in 3.6. The mentioned change also helps me to fix openstack ansible tests under python 3.6 and seems to be ok from my POV

[gtema]

PR #40341

[gtema]

for reference: ansible 2.5 backport PR #40379