ERROR! Unexpected Exception, this is probably a bug: Could not load "is": 'is'

[MrDrMcCoy]

Summary

When I attempt to run my playbook that updates host information in Netbox, I receive the following error:

ERROR! Unexpected Exception, this is probably a bug: Could not load "is": 'is'
  line 0

This appears while running different tasks, depending on the tags I run with. Sometimes it appears when running netbox.netbox.netbox_device, and other times with ansible.builtin.uri. Current playbook/role and log with -vvv is attached.

ansible.log.gz

Issue Type

Bug Report

Component Name

jinja2

Ansible Version

This issue was able to be reproduced on Fedora 38 (dnf):

❯ ansible --version
ansible [core 2.14.5]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/jmccoy/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jmccoy/.local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/jmccoy/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jmccoy/.local/bin/ansible
  python version = 3.11.3 (main, Apr  5 2023, 00:00:00) [GCC 13.0.1 20230401 (Red Hat 13.0.1-0)] (/usr/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

as well as on OpenSUSE Tumbleweed (distrobox/pip):

$ ansible --version
ansible [core 2.14.5]
  config file = None
  configured module search path = ['/home/jmccoy/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jmccoy/.local/lib/python3.11/site-packages/ansible
  ansible collection location = /home/jmccoy/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/jmccoy/.local/bin/ansible
  python version = 3.11.2 (main, Mar 03 2023, 18:48:37) [GCC] (/usr/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

Configuration

$ ansible-config dump --only-changed -t all
BECOME_PASSWORD_FILE(env: ANSIBLE_BECOME_PASSWORD_FILE) = /home/jmccoy/.pas
CONFIG_FILE() = None
DEFAULT_FORKS(env: ANSIBLE_FORKS) = 16
DEFAULT_LOG_PATH(env: ANSIBLE_LOG_PATH) = /home/jmccoy/.cache/ansible/ansible.log
DEFAULT_NO_TARGET_SYSLOG(env: ANSIBLE_NO_TARGET_SYSLOG) = False
DEFAULT_STDOUT_CALLBACK(env: ANSIBLE_STDOUT_CALLBACK) = yaml
DISPLAY_ARGS_TO_STDOUT(env: ANSIBLE_DISPLAY_ARGS_TO_STDOUT) = True

OS / Environment

Fedora 38 (dnf) and OpenSUSE Tumbleweed (distrobox/pip).

Steps to Reproduce

ansible-playbook -e token=REDACTED -i hosts netbox_update.yml --limit=production_cassandra --tags=address

netbox_update.tar.gz

Expected Results

I expect this to either work or produce an actionable error message as opposed to a cryptic stack trace.

Actual Results

2023-05-10 18:47:36,341 p=56774 u=jmccoy n=ansible | The full traceback is:
Traceback (most recent call last):
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/template/__init__.py", line 445, in __getitem__
    func = self._delegatee[key]
           ~~~~~~~~~~~~~~~^^^^^
KeyError: 'is'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/executor/task_executor.py", line 111, in run
    items = self._get_loop_items()
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/executor/task_executor.py", line 249, in _get_loop_items
    items = templar.template(self._task.loop)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/template/__init__.py", line 724, in template
    result = self.do_template(
             ^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/template/__init__.py", line 987, in do_template
    res = self.environment.concat(rf)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/template/native_helpers.py", line 43, in ansible_eval_concat
    head = list(islice(nodes, 2))
           ^^^^^^^^^^^^^^^^^^^^^^
  File "<template>", line 18, in root
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/template/__init__.py", line 264, in wrapper
    return list(ret)
           ^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/jinja2/filters.py", line 1765, in select_or_reject
    if func(item):
       ^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/jinja2/filters.py", line 1750, in <lambda>
    return lambda item: modfunc(func(transfunc(item)))
                                ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/jinja2/filters.py", line 1745, in func
    return context.environment.call_test(name, item, args, kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/jinja2/environment.py", line 589, in call_test
    return self._filter_test_common(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/jinja2/environment.py", line 510, in _filter_test_common
    func = env_map.get(name)  # type: ignore
           ^^^^^^^^^^^^^^^^^
  File "<frozen _collections_abc>", line 774, in get
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/template/__init__.py", line 447, in __getitem__
    raise TemplateSyntaxError('Could not load "%s": %s' % (key, to_native(original_exc or e)), 0)
jinja2.exceptions.TemplateSyntaxError: Could not load "is": 'is'
  line 0

2023-05-10 18:47:36,341 p=56774 u=jmccoy n=ansible | fatal: [cass-005-0011]: FAILED! => 
  msg: |-
    Unexpected failure during module execution: Could not load "is": 'is'
      line 0
  stdout: ''

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibot]

Files identified in the description:

• lib/ansible/template

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[sivel]

This error indicates that you are attempting to use a is as a filter somewhere in your variables.

An example that causes this:

$ ansible localhost -m debug -a 'msg={{name | is}}'
localhost | FAILED! => {
    "msg": "template error while templating string: Could not load \"is\": 'is'. String: {{\"foo\"|is}}. Could not load \"is\": 'is'"
}

The is keyword in jinja2 is supposed to be used like the following in test syntax:

name is defined

So you would likely need to grep or otherwise search for something to the effect of:

$ egrep -HRn '\|\s*is\b' *
80762.yml:5:        msg: '{{ name | is }}'

[MrDrMcCoy]

@sivel Thank you for your fast response! However, grepping for that in my role comes up dry:

❯ grep -PHRn '\|\s*is' roles/netbox_update
❌1 02:01pm 05/10 [jmccoy-netbox] (M8) jmccoy@fedora:…/configMgmt/ansible
❯

To the best of my knowledge, I have not used the is filter in that way anywhere in this role. Broadening the search yields a few instances where it appears to be used correctly:

❯ grep -PHRn '\sis\s' roles/netbox_update
roles/netbox_update/tasks/address.yml:2:## This is required because the netbox module cannot fix incorrect netmasks
roles/netbox_update/tasks/address.yml:61:## This is required because the netbox module cannot fix incorrect netmasks
roles/netbox_update/tasks/address.yml:134:## This is required because the netbox module cannot fix incorrect netmasks
roles/netbox_update/tasks/interface.yml:30:          or ('Virtual' if item.module is defined and 'virtio' in item.module)
roles/netbox_update/tasks/interface.yml:53:  when: ansible_facts.ansible_local.bz_ipmi is defined
roles/netbox_update/tasks/main.yml:4:# the device update step sometimes fails with an error about not being able to resolve the id of a primary ip. this is due to the prefix update drifting due to unknown reasons. re-running this role multiple times will eventually fix it. this keeps happening, and needs to be resolved somehow.
roles/netbox_update/tasks/netbox_query.yml:76:  ignore_errors: true # otherwise fails if host is not existing
roles/netbox_update/tasks/add_host.yml:2:- name: Ensure host is not a VM

Replacing thing is defined with ( thing | default(None) ) does not improve the situation. I receive the same error even when the is test is removed from every part of this role. Any other ideas?

[sivel]

The issue seems to be with this task:

netbox_update : Patch Netbox with correct IPMI IPv4 address by talking directly to API

Looking at the traceback, it indicates the error is happening in the process of templating the loop: on that task:

Traceback (most recent call last):
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/executor/task_executor.py", line 111, in run
    items = self._get_loop_items()
            ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmccoy/.local/lib/python3.11/site-packages/ansible/executor/task_executor.py", line 249, in _get_loop_items
    items = templar.template(self._task.loop)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Can you please provide us with that task definition including the loop?

I've played around with this a bit more, and something like this must be in there:

loop: '{{ things|select("is") }}'

select could be one of select, selectattr, reject, or rejectattr I imagine.

Maybe something like this:

$ egrep -HRn '\|\s*(select|reject)(attr)?.*is'
./80762.yml:6:      loop: '{{ things|select("is") }}'

[MrDrMcCoy]

@sivel Thank you for identifying that the error came from that loop. I was able to find the unmatched 'is' and work around the error by changing selectattr('family.value', 'is', 4) to selectattr('family.value', '==', 4).

Although this allows me to proceed, I find the matter perplexing. The Jinja2 docs for selectattr() suggest that this is the correct way to invoke this sort of test, and it works with 'in' as one might expect. The Ansible docs suggest that is is an Ansible-specific test that is supposed to be registered for use in Jinja filters. Have we discovered an edge case where the test is not properly imported?

[sivel]

Can you point to the docs that suggest you can use is in that manner? is isn't a test itself, it is a keyword to instruct jinja2 to compare a value using a test. is isn't the same as eq or ==.

From the upstream jinja2 docs:

Tests can be used to test a variable against a common expression. To test a variable or expression, you add is plus the name of the test after the variable

If our docs are misleading, we should correct that.

[MrDrMcCoy]

My understanding came from: https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_tests.html#test-syntax

Historically Ansible has registered tests as both jinja tests and jinja filters, allowing for them to be referenced using filter syntax.

As of Ansible 2.5, using a jinja test as a filter will generate a deprecation warning. As of Ansible 2.9+ using jinja test syntax is required.

The syntax for using a jinja test is as follows

variable is test_name

The usage of is throughout the examples led me to believe that it was a test that could be used as part of Jinja filters. However, I now see in my current reading that is precedes the actual test name in each example. The error was on me from top to bottom, so I can only thank you for your help and close this as not a bug.