JMESPathError in json_query filter plugin in v2.9.23

[BrainStone]

Summary

When using a json_query filter after updating to 2.9.23 from 2.9.21 (can't 100% pinpoint this but I believe the critical change happened when updating from v2.9.22 to v2.9.23) I get this error:

JMESPathError in json_query filter plugin:
In function ends_with(), invalid type for value: config.yml.j2, expected one of: ['string'], received: "unknown"

Seems to be a recurrance of ansible/ansible#27299

Issue Type

Bug Report

Component Name

json_query

Ansible Version

$ ansible --version
ansible 2.9.23
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible
  python version = 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]

Configuration

$ ansible-config dump --only-changed

OS / Environment

Debian 10

Steps to Reproduce

Expression that triggers it:

{{ local_config_files | json_query('[? state == `file` && ends_with(path, `.j2`)]') }}

Example object for local_config_files:

local_config_files:
  - state: file
    path: test.yml
  - state: file
    path: test.yml.j2
  - state: directory
    path: foobar

Expected Results

To work like it did in v2.9.21, aka that json_query call working as expected with the given input.

Actual Results

TASK [plugin_template : Template AdvancedBan configs] ********************************************************************************
fatal: [proxy]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nIn function ends_with(), invalid type for value: config.yml.j2, expected one of: ['string'], received: \"unknown\""}

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/filter/json_query.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[BrainStone]

The workaround presented in the original issue still works btw:

Broken:

{{ local_config_files | json_query('[? state == `file` && ends_with(path, `.j2`)]') }}

Working:

{{ local_config_files | to_json | from_json | json_query('[? state == `file` && ends_with(path, `.j2`)]') }}

[felixfontein]

You are apparently using the filter included in Ansible 2.9, and not the one from this repository. In any case, the filter itself in Ansible 2.9 did not change (https://github.com/ansible/ansible/blob/stable-2.9/lib/ansible/plugins/filter/json_query.py was last changed in 2017). Please note that Ansible 2.9 is in maintenance mode and only receives security fixes, so you likely will have to install community.general and use the filter from there.

[BrainStone]

Interesting that security fixes introduce regressions.

I'd be happy to report the issue there if the bot would let me: ansible/ansible#75091

Also pretty sure I have it installed, not sure what exactly changed that it suddely broke. And the reason for me using this version is simply that that's the version being shipped on Debian 10.

[felixfontein]

Are you actually sure that it worked with 2.9.21? I did run your reproducer with Ansible 2.9.20, 2.9.21, 2.9.22 and 2.9.23, and it always resulted in fatal: [localhost]: FAILED! => {"msg": "JMESPathError in json_query filter plugin:\nIn function ends_with(), invalid type for value: test.yml, expected one of: ['string'], received: \"unknown\""}.

[felixfontein]

Debian 10 comes with Ansible 2.7.7, btw (https://packages.debian.org/buster/ansible).

[felixfontein]

Your reproducer works fine with the version of json_query included in this repo, btw. (Which is not surprising, since it has been fixed in #320.)

[BrainStone]

Right. Now that you mention it. I actually use the version on http://ppa.launchpad.net/ansible/ansible/ubuntu trusty. As 2.7 was too old for my use case.

# apt-cache policy ansible
ansible:
  Installed: 2.9.23-1ppa~trusty
  Candidate: 2.9.23-1ppa~trusty
  Version table:
 *** 2.9.23-1ppa~trusty 500
        500 http://ppa.launchpad.net/ansible/ansible/ubuntu trusty/main amd64 Packages
        100 /var/lib/dpkg/status
     2.9.16+dfsg-1~bpo10+2 100
        100 http://deb.debian.org/debian buster-backports/main amd64 Packages
     2.7.7+dfsg-1 500
        500 http://deb.debian.org/debian buster/main amd64 Packages

[aminvakil]

@BrainStone Can you confirm issue has been fixed using json_query filter from community.general collection?

needs_info

[BrainStone]

Yes the community.general version works as expected.

So as it seems that this issue doesn't belong here it would be very much aprechiated if my issue (ansible/ansible#75091) over on the Ansible repo could be reopened after it has been closed by a bot (and I can't reopen it)