aws_secret lookup doesn't honor on_missing when using nested

[gionn]

Summary

I am using multiple lookup calls to populate secret variables from a single AWS secret containing multiple (nested) secrets, however it seems that on_missing applies only to the aws secret itself while I was hoping to get the same behaviour only when the nested key is missing inside the aws secret.

Issue Type

Bug Report

Component Name

amazon.aws.aws_secret

Ansible Version

$ ansible --version
ansible [core 2.14.4]
  config file = /Users/Giovanni.Toraldo/src/alfresco/alfresco-ansible-deployment/ansible.cfg
  configured module search path = ['/Users/Giovanni.Toraldo/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/Giovanni.Toraldo/.virtualenvs/alfresco-ansible-deployment-LdMEq9P-/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/Giovanni.Toraldo/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/Giovanni.Toraldo/.virtualenvs/alfresco-ansible-deployment-LdMEq9P-/bin/ansible
  python version = 3.10.12 (main, Jul 28 2023, 18:44:44) [Clang 14.0.3 (clang-1403.0.22.14.1)] (/Users/Giovanni.Toraldo/.virtualenvs/alfresco-ansible-deployment-LdMEq9P-/bin/python)
  jinja version = 3.1.2
  libyaml = True

Collection Versions

$ ansible-galaxy collection list
Collection                     Version
------------------------------ -------
amazon.aws                     6.3.0  
ansible.posix                  1.5.4  
ansible.utils                  2.6.0  
community.aws                  6.3.0  
community.crypto               2.10.0 
community.docker               3.4.8  
community.general              7.4.0  
community.postgresql           2.1.0  
middleware_automation.common   1.1.2  
middleware_automation.keycloak 1.3.0  

AWS SDK versions

$ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.9
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/Giovanni.Toraldo/.virtualenvs/alfresco-ansible-deployment-LdMEq9P-/lib/python3.10/site-packages
Requires: botocore, jmespath, s3transfer
Required-by: 
---
Name: botocore
Version: 1.29.165
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email: 
License: Apache License 2.0
Location: /Users/Giovanni.Toraldo/.virtualenvs/alfresco-ansible-deployment-LdMEq9P-/lib/python3.10/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer

Configuration

$ ansible-config dump --only-changed

OS / Environment

Ubuntu

Steps to Reproduce

sync_db_password: "{{ lookup('amazon.aws.aws_secret', user_provided_project_name + '.sync_db_password', on_missing='warn', nested=true) }}"
identity_admin_password: "{{ lookup('amazon.aws.aws_secret', user_provided_project_name + '.identity_admin_password', on_missing='warn', nested=true) }}"

Expected Results

Expecting just a warning if the secret exists but doesn't contain identity_admin_password

Actual Results

fatal: [localhost]: FAILED! => {"msg": "An unhandled exception occurred while templating '{'sync_db_password': \"{{ lookup('amazon.aws.aws_secret', user_provided_project_name + '.sync_db_password', on_missing='warn', nested=true) }}\", 'identity_admin_password': \"{{ lookup('amazon.aws.aws_secret', user_provided_project_name + '.identity_admin_password', on_missing='warn', nested=true) }}\"}'. Error was a <class 'ansible.errors.AnsibleLookupError'>, original message: Successfully retrieved secret but there exists no key identity_admin_password in the secret"}

Code of Conduct

• I agree to follow the Ansible Code of Conduct

[abikouo]

@gionn Thanks for raising the issue, I have created a Pull request to fix it. Could you please give a try with #1972
Thanks