Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

include_filters without filters returns ALL hosts #457

Closed
1 task done
chrismeyersfsu opened this issue Aug 11, 2021 · 1 comment · Fixed by #456
Closed
1 task done

include_filters without filters returns ALL hosts #457

chrismeyersfsu opened this issue Aug 11, 2021 · 1 comment · Fixed by #456
Assignees
@goneri
Labels
bug This issue/PR relates to a bug has_pr inventory inventory plugin jira plugins plugin (any type) python3

Comments

@chrismeyersfsu
Copy link
Contributor

Summary

include_filters is matching on ALL hosts when I expect it to match 0 hosts.

I think what is happening is that filters is the initial filter that is being applied and then include_filters are being applied to that set of matching hosts. If you don't specify ANY filters then it's equivalent to True as the initial filter. Then, when include_filters gets applied you get something like True || include_filter_1 || include_filter_2. Effectively, the include_filters become a NOOP if filters is not specified. I confirmed this suspicion with some pdb in the code.

amazon.aws/plugins/inventory/aws_ec2.py

Line 712 in 5876c14

include_filters = [self.get_option('filters')] + self.get_option('include_filters')

include_filters = [self.get_option('filters')] + self.get_option('include_filters') <-- in my debugging this is [{}, {'key-name': 'jenkins'}]. Notice the first {}, when the filter list is passed to ansible_dict_to_boto3_filter_list(filter) the empty dict is equivalent to True.

#456 I have a potential fix here, but I feel like it may not take into account the exclude_filters feature.

Issue Type

Bug Report

Component Name

aws_ec2

Ansible Version

$ ansible --version
ansible [core 2.11.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /var/lib/awx/venv/awx/lib64/python3.8/site-packages/ansible
  ansible collection location = /var/lib/awx/.ansible/collections:/usr/share/ansible/collections
  executable location = /var/lib/awx/venv/awx/bin/ansible
  python version = 3.8.6 (default, Jan 29 2021, 17:38:16) [GCC 8.4.1 20200928 (Red Hat 8.4.1-1)]
  jinja version = 2.11.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0
ansible.netcommon             2.3.0
ansible.posix                 1.2.0
ansible.utils                 2.3.1
ansible.windows               1.7.2
arista.eos                    2.2.0
awx.awx                       19.2.2
azure.azcollection            1.8.0
check_point.mgmt              2.0.0
chocolatey.chocolatey         1.1.0
cisco.aci                     2.0.0
cisco.asa                     2.0.2
cisco.intersight              1.0.16
cisco.ios                     2.3.1
cisco.iosxr                   2.4.0
cisco.meraki                  2.4.2
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.5.0
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.0.0
community.crypto              1.8.0
community.digitalocean        1.8.0
community.docker              1.9.0
community.fortios             1.0.0
community.general             3.5.0
community.google              1.0.0
community.grafana             1.2.1
community.hashi_vault         1.3.2
community.hrobot              1.1.1
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.0
community.mysql               2.1.0
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.4.0
community.proxysql            1.1.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.12.0
community.windows             1.6.0
community.zabbix              1.4.0
containers.podman             1.6.2
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.11.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.1
google.cloud                  1.0.2
hetzner.hcloud                1.4.4
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.2.0
junipernetworks.junos         2.4.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.8.1
netapp.cloudmanager           21.9.0
netapp.elementsw              21.6.1
netapp.ontap                  21.9.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.1.1
ngine_io.cloudstack           2.1.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.0
openvswitch.openvswitch       2.0.0
ovirt.ovirt                   1.5.4
purestorage.flasharray        1.10.0
purestorage.flashblade        1.6.0
sensu.sensu_go                1.11.1
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.20.0
theforeman.foreman            2.1.2
vyos.vyos                     2.5.0
wti.remote                    1.0.1

# /var/lib/awx/venv/awx/lib64/python3.8/site-packages/ansible_collections
Collection                    Version
----------------------------- -------
amazon.aws                    1.5.0
ansible.netcommon             2.3.0
ansible.posix                 1.2.0
ansible.utils                 2.3.1
ansible.windows               1.7.2
arista.eos                    2.2.0
awx.awx                       19.2.2
azure.azcollection            1.8.0
check_point.mgmt              2.0.0
chocolatey.chocolatey         1.1.0
cisco.aci                     2.0.0
cisco.asa                     2.0.2
cisco.intersight              1.0.16
cisco.ios                     2.3.1
cisco.iosxr                   2.4.0
cisco.meraki                  2.4.2
cisco.mso                     1.2.0
cisco.nso                     1.0.3
cisco.nxos                    2.5.0
cisco.ucs                     1.6.0
cloudscale_ch.cloud           2.2.0
community.aws                 1.5.0
community.azure               1.0.0
community.crypto              1.8.0
community.digitalocean        1.8.0
community.docker              1.9.0
community.fortios             1.0.0
community.general             3.5.0
community.google              1.0.0
community.grafana             1.2.1
community.hashi_vault         1.3.2
community.hrobot              1.1.1
community.kubernetes          1.2.1
community.kubevirt            1.0.0
community.libvirt             1.0.2
community.mongodb             1.3.0
community.mysql               2.1.0
community.network             3.0.0
community.okd                 1.1.2
community.postgresql          1.4.0
community.proxysql            1.1.0
community.rabbitmq            1.1.0
community.routeros            1.2.0
community.skydive             1.0.0
community.sops                1.1.0
community.vmware              1.12.0
community.windows             1.6.0
community.zabbix              1.4.0
containers.podman             1.6.2
cyberark.conjur               1.1.0
cyberark.pas                  1.0.7
dellemc.enterprise_sonic      1.1.0
dellemc.openmanage            3.6.0
dellemc.os10                  1.1.1
dellemc.os6                   1.0.7
dellemc.os9                   1.0.4
f5networks.f5_modules         1.11.0
fortinet.fortimanager         2.1.3
fortinet.fortios              2.1.2
frr.frr                       1.0.3
gluster.gluster               1.0.1
google.cloud                  1.0.2
hetzner.hcloud                1.4.4
hpe.nimble                    1.1.3
ibm.qradar                    1.0.3
infinidat.infinibox           1.2.4
inspur.sm                     1.2.0
junipernetworks.junos         2.4.0
kubernetes.core               1.2.1
mellanox.onyx                 1.0.0
netapp.aws                    21.6.0
netapp.azure                  21.8.1
netapp.cloudmanager           21.9.0
netapp.elementsw              21.6.1
netapp.ontap                  21.9.0
netapp.um_info                21.7.0
netapp_eseries.santricity     1.2.13
netbox.netbox                 3.1.1
ngine_io.cloudstack           2.1.0
ngine_io.exoscale             1.0.0
ngine_io.vultr                1.1.0
openstack.cloud               1.5.0
openvswitch.openvswitch       2.0.0
ovirt.ovirt                   1.5.4
purestorage.flasharray        1.10.0
purestorage.flashblade        1.6.0
sensu.sensu_go                1.11.1
servicenow.servicenow         1.0.6
splunk.es                     1.0.2
t_systems_mms.icinga_director 1.20.0
theforeman.foreman            2.1.2
vyos.vyos                     2.5.0
wti.remote                    1.0.1

# /var/lib/awx/.ansible/collections/ansible_collections
Collection Version
---------- -------
amazon.aws 1.5.0

AWS SDK versions

$ pip show boto boto3 botocore
Name: boto
Version: 2.49.0
Summary: Amazon Web Services Library
Home-page: https://github.com/boto/boto/
Author: Mitch Garnaat
Author-email: mitch@garnaat.com
License: MIT
Location: /var/lib/awx/venv/awx/lib/python3.8/site-packages
Requires:
Required-by:
---
Name: boto3
Version: 1.18.18
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /var/lib/awx/venv/awx/lib/python3.8/site-packages
Requires: botocore, s3transfer, jmespath
Required-by:
---
Name: botocore
Version: 1.21.18
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: None
License: Apache License 2.0
Location: /var/lib/awx/venv/awx/lib/python3.8/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: s3transfer, boto3

Configuration

$ ansible-config dump --only-changed

OS / Environment

CentOS Linux release 8.4.2105

Steps to Reproduce

---
plugin: amazon.aws.aws_ec2
regions:
  - us-east-1
  - us-east-2
strict_permissions: False
include_filters:
- key-name: "does_not_exist"

ansible-inventory -i aws_ec2.yml --list

  • Do NOT use the filters: feature.
  • Make sure the include_filters does NOT match anything in your EC2 infrastructure.

Expected Results

I expected 0 hosts to be returned because I assumed that at least 1 filter in the include_filters list had to match.

Actual Results

  • All hosts in my EC2 infrastructure were returned.

Code of Conduct

  • I agree to follow the Ansible Code of Conduct
@chrismeyersfsu chrismeyersfsu mentioned this issue Aug 11, 2021
Merged
@ansibullbot
Copy link

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibullbot ansibullbot added bug This issue/PR relates to a bug has_pr inventory inventory plugin needs_triage plugins plugin (any type) python3 labels Aug 11, 2021
@goneri goneri self-assigned this Aug 24, 2021
@goneri goneri added the jira label Aug 24, 2021
ansible-zuul bot pushed a commit that referenced this issue Aug 31, 2021
@chrismeyersfsu
do not include {} boto filter by default (#456)
05d1ae0 
do not include {} boto filter by default

Depends-On: #480

When no filter: specified, do not include an empty dict in the list of
boto filters because it effectively matches any host.

Closes: #457
Closes: #452
ISSUE TYPE

Bugfix Pull Request

Reviewed-by: Mark Chappell <None>
Reviewed-by: Alina Buzachis <None>
Reviewed-by: None <None>
abikouo pushed a commit to abikouo/amazon.aws that referenced this issue Oct 24, 2023
@goneri
add coverage as a test requirement (ansible-collections#457)
ce7ccf8 
Add the `coverage` module in the test-requirements.txt file. The zuul
unit-tests job depends on it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@goneri goneri

Labels
bug This issue/PR relates to a bug has_pr inventory inventory plugin jira plugins plugin (any type) python3
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

do not include {} boto filter by default chrismeyersfsu/amazon.aws
4 participants
@goneri @chrismeyersfsu @ansibullbot @abikouo