ansible-lint fails after upgrade to 4.1.1

[g1ps]

Issue Type

• Bug report

After upgrading to 4.1.1 I get this error on every run. I tried 4.1.1.a0, and reverting back through 4.1.0 to 4.0.1. No change. I also noted that the upgrade pused ansible to 2.8.3, from my previously installed 2.8.2 so I reverted that as well. Nothing has helped. I suspect something in the dependencies might have broken it but I don't know what they were. My builds are now broken due to failing lint checks and I am so far unable to fix this.

Ansible and Ansible Lint details

$ ansible-lint --version
ansible-lint 4.0.1
$ ansible --version
ansible 2.8.2
  config file = None
  configured module search path = [u'/users/do-auto-user/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.5 (default, Jun 11 2019, 12:19:05) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]

• ansible installation method: pip
• ansible-lint installation method: pip

Desired Behaviour

No error.

Actual Behaviour (Bug report only)

Traceback (most recent call last):
  File "/usr/bin/ansible-lint", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/site-packages/ansiblelint/__main__.py", line 187, in main
    matches.extend(runner.run())
  File "/usr/lib/python2.7/site-packages/ansiblelint/__init__.py", line 264, in run
    for child in ansiblelint.utils.find_children(arg, self.playbook_dir):
  File "/usr/lib/python2.7/site-packages/ansiblelint/utils.py", line 171, in find_children
    for child in play_children(basedir, item, playbook[1], playbook_dir):
  File "/usr/lib/python2.7/site-packages/ansiblelint/utils.py", line 222, in play_children
    fail_on_undefined=False)
  File "/usr/lib/python2.7/site-packages/ansiblelint/utils.py", line 190, in template
    **dict(kwargs, fail_on_undefined=fail_on_undefined))
  File "/usr/lib/python2.7/site-packages/ansiblelint/utils.py", line 78, in ansible_template
    return templar.template(varname, **kwargs)
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 572, in template
    ) for v in variable]
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 584, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 584, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 539, in template
    disable_lookups=disable_lookups,
  File "/usr/lib/python2.7/site-packages/ansible/template/__init__.py", line 804, in do_template
    res = j2_concat(rf)
  File "<template>", line 13, in root
  File "/usr/lib/python2.7/site-packages/ansible/plugins/filter/core.py", line 65, in to_yaml
    transformed = yaml.dump(a, Dumper=AnsibleDumper, allow_unicode=True, default_flow_style=default_flow_style, **kw)
  File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 293, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all
    dumper.represent(data)
  File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 29, in represent
    node = self.represent_data(data)
  File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 68, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/usr/lib64/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', AnsibleUndefined)

[webknjaz]

pip freeze before and after would be helpful. Also: pin your traversed dependency versions.

[g1ps]

I don't know what that means but I'll investigate. Thanks.

[g1ps]

4.1.0 also will not install on a clean RHEL7 VM.

sudo pip install ansible-lint==4.1.0

    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python2 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-r0nNN3/ruamel.yaml/setup.py'"'"'; __file__='"'"'/tmp/pip-install-r0nNN3/ruamel.yaml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-r0nNN3/ruamel.yaml/
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-r0nNN3/ruamel.yaml/setup.py", line 23, in <module>
        from setuptools.namespaces import Installer as NameSpaceInstaller # NOQA
    ImportError: No module named namespaces
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[webknjaz]

Same problem, same solution + anyway it's failing to install ruamel, not ansible-lint.

[webknjaz]

I mean same as #565 (comment)

[g1ps]

I know but it's a dependency which never failed with 4.0. I suspect it's a new one, is it? Either way, it breaks ansible-lint.

[g1ps]

-sh-4.2$ pip freeze
ansible==2.8.2
ansible-lint==4.0.1
asn1crypto==0.24.0
backports.ssl-match-hostname==3.5.0.1
cffi==1.12.3
chardet==2.2.1
configobj==4.7.2
cryptography==2.7
decorator==3.4.0
enum34==1.1.6
ethtool==0.8
iniparse==0.4
ipaddress==1.0.16
Jinja2==2.10.1
kitchen==1.1.1
lxml==3.2.1
M2Crypto==0.21.1
Magic-file-extensions==0.2
MarkupSafe==1.1.1
pathspec==0.5.9
pciutils==1.7.3
perf==0.1
pycparser==2.19
pycurl==7.19.0
pygobject==3.22.0
pygpgme==0.3
pyinotify==0.9.4
pyliblzma==0.5.3
pyOpenSSL==0.13.1
python-dateutil==1.5
python-dmidecode==3.10.13
python-linux-procfs==0.4.9
pyudev==0.15
pyxattr==0.5.1
PyYAML==5.1.2
rhnlib==2.5.65
schedutils==0.4
six==1.9.0
slip==0.4.0
slip.dbus==0.4.0
SSSDConfig==1.16.0
subscription-manager==1.20.11
urlgrabber==3.10
yamllint==1.16.0
yum-metadata-parser==1.1.4

[webknjaz]

So the issue is that ruamel.yaml released a new version recently and only uploaded sdist but not wheel: https://pypi.org/project/ruamel.yaml/0.16.1/#files
The previous version is fine. So you need to complain about it somewhere @ ruamel's bug tracker.

As a workaround use

pip install --user --only-binary ruamel.yaml ansible-lint

or

pip install --user --prefer-binary ansible-lint

Ultimate/universal solution that prevents similar future problems:

Use a separate virtualenv for ansible-lint and use pip-tools to generate a constraints file with exact versions of all indirect dependencies with dist hashes.

[webknjaz]

Upstream issue: https://bitbucket.org/ruamel/yaml/issues/303/publish-a-wheel-for-0161.

[g1ps]

As a workaround use...

OK, thanks, that's good to know.

[AvdN]

The .whl for 0.16.1 just got uploaded

[sergey-shuyskiy]

python --version
Python 2.7.5

test.yml

- hosts: localhost
  tasks:
    - name: test task
      set_fact:
        test_variable: "{{ test_variable2 | to_nice_yaml }}"

steps to reproduce the issue:

virtualenv venv-ansible-lint
source venv-ansible-lint/bin/activate
pip install ansible-lint==4.1.0
ansible-lint test.yml

error:

Traceback (most recent call last):
  File "/home/master/venv-ansible-lint/bin/ansible-lint", line 8, in <module>
    sys.exit(main())
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansiblelint/__main__.py", line 187, in main
    matches.extend(runner.run())
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansiblelint/__init__.py", line 264, in run
    for child in ansiblelint.utils.find_children(arg, self.playbook_dir):
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansiblelint/utils.py", line 171, in find_children
    for child in play_children(basedir, item, playbook[1], playbook_dir):
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansiblelint/utils.py", line 222, in play_children
    fail_on_undefined=False)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansiblelint/utils.py", line 190, in template
    **dict(kwargs, fail_on_undefined=fail_on_undefined))
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansiblelint/utils.py", line 78, in ansible_template
    return templar.template(varname, **kwargs)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansible/template/__init__.py", line 572, in template
    ) for v in variable]
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansible/template/__init__.py", line 584, in template
    disable_lookups=disable_lookups,
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansible/template/__init__.py", line 584, in template
    disable_lookups=disable_lookups,
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansible/template/__init__.py", line 539, in template
    disable_lookups=disable_lookups,
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansible/template/__init__.py", line 804, in do_template
    res = j2_concat(rf)
  File "<template>", line 12, in root
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/ansible/plugins/filter/core.py", line 71, in to_nice_yaml
    transformed = yaml.dump(a, Dumper=AnsibleDumper, indent=indent, allow_unicode=True, default_flow_style=False, **kw)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/yaml/__init__.py", line 293, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/yaml/__init__.py", line 281, in dump_all
    dumper.represent(data)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/yaml/representer.py", line 29, in represent
    node = self.represent_data(data)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/yaml/representer.py", line 68, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/home/master/venv-ansible-lint/lib/python2.7/site-packages/yaml/representer.py", line 251, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', AnsibleUndefined)

pip freeze
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ansible==2.8.6
ansible-lint==4.1.0
cffi==1.13.1
cryptography==2.8
enum34==1.1.6
ipaddress==1.0.23
Jinja2==2.10.3
MarkupSafe==1.1.1
pycparser==2.19
PyYAML==5.1.2
ruamel.ordereddict==0.4.14
ruamel.yaml==0.16.5
ruamel.yaml.clib==0.2.0
six==1.12.0

it solved the issue for me:
pip install PyYAML==3.13

[g1ps]

Seems to be working now on a clean isntallation using 4.1.0 and 4.1.1a2. Thanks.