Dynamic Inventory Source Fails to Load Hosts from EC2 Plugin (awx-operator installed on AWS EKS Cluster)

[gilbertorobles24]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that the AWX Operator is open source software provided for free and that I might not receive a timely response.

Bug Summary

I created a Terraform infrastructure to deploy an EKS cluster on AWS, on which I followed the recommended installation guide from the AWX operator documentation. The server runs well, I can log in, I can connect to GitLab Integrations to dynamically retrieve playbooks. But when attempting to sync my inventory sources through the EC2 plugin (aws_ec2), using correct credentials, I get 0 hosts returned. I can confirm my Terraform infrastructure has open security groups, and the target EC2 instances are part of the same VPC, sharing the same network as the AWX server. I also can successfully SSH into the EC2 instances from the AWX server, as well as successfully retrieve instance information with the AWS CLI from the same node, but my inventory does not seem to be able to connect at all to AWS.

AWX Operator version

1.1.3

AWX version

21.11.0

Kubernetes platform

other (please specify in additional information)

Kubernetes/Platform version

AWS EKS

Modifications

yes

Steps to reproduce

Terraform ~> 4.0
Kubernetes ~> 2.16
EKS version 1.24

TERRAFORM / KUBERNETES
Simple terraform infrastructure provisioning a VPC with an EKS cluster, and a target EC2 instance. With open security groups, sharing the same network.

AWX INSTALLATION
Standard installation. Following exactly the steps from the awx-operator installation guide documentation.

AWX SETUP
Simple vanilla organization, vanilla project. Connected to GitLab via SSH to retrieve playbooks (not used or required to replicate this issue).
Vanilla Inventory, using Amazon EC2 plugin as dynamic source.

Expected results

Upon running a dynamic inventory source sync with the AWS EC2 plugin, inventory should populate Hosts with available EC2 target instance public ip address.

Actual results

setting up inventory plugins
redirecting (type: inventory) ansible.builtin.aws_ec2 to amazon.aws.aws_ec2
Loading collection amazon.aws from /usr/share/ansible/collections/ansible_collections/amazon/aws
Parsed /runner/inventory/aws_ec2.yml inventory source with auto plugin

   23.308 INFO     Processing JSON output...
   23.309 DEBUG    Loaded group: all
   23.311 DEBUG    Loaded group: aws_ec2
   23.312 DEBUG    Adding child group aws_ec2 to parent all
   23.312 INFO     Loaded 1 groups, 0 hosts
   23.313 DEBUG    Removing empty group aws_ec2
   23.336 DEBUG    Inventory variables unmodified
   23.358 INFO     Inventory import completed for ec2-dynamic-inventory in 0.1s

Additional information

During my installation, I had to manually configure a yaml file for a missing persistent volume. The PVC was configured, but I had to manually create the PV. Potentially another bug of its own. I followed the installation instructions exactly.

Additionally, all my update options are checked, [overwrite variables, update on launch].

Reinstalled boto3/botocore libraries on node hosting AWX server.

Operator Logs

No response

[0x7081]

How many hosts are you trying to import?
I had a pretty similar error with my AWX instance that's hosted on EKS, but my issue was with the script plugin.

You could check my solution in ansible/awx#13396 (comment), maybe this helps.

[fosterseth]

Hello @gilbertorobles24 , this appears to be less of a bug report or feature request and more of a question.
Could you please ask this on our mailing list? See https://github.com/ansible/awx/#get-involved for information for ways to connect with us.