Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

route53_facts can't return a list of max_items > 100 #49457

Closed
IPvSean opened this issue Dec 3, 2018 · 9 comments · Fixed by #60433 · May be fixed by #31860

Comments

@IPvSean
Copy link
Contributor

@IPvSean IPvSean commented Dec 3, 2018

SUMMARY

The route53_facts offers the parameter max_items: 200 but it seems to max out at 100 no matter what? Should either
A) allow us to go bigger than 100
OR
B) error and tell us we can't go bigger than 100 and document this is an AWS limitation rather than an Ansible one

ISSUE TYPE
  • Bug Report
COMPONENT NAME

route53_facts

ANSIBLE VERSION
ansible 2.7.4
  config file = /Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg
  configured module search path = [u'/Users/sean/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Python/2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.10 (default, Aug 17 2018, 17:41:52) [GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.0.42)]
CONFIGURATION
DEFAULT_CALLBACK_WHITELIST(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = [u'time']
DEFAULT_FORKS(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = 50
DEFAULT_HOST_LIST(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = [u'/Users/sean/Documents/GitHub/linklight/provisioner/hosts']
DEFAULT_NO_TARGET_SYSLOG(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = False
DEFAULT_SCP_IF_SSH(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = True
HOST_KEY_CHECKING(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = False
RETRY_FILES_ENABLED(/Users/sean/Documents/GitHub/linklight/provisioner/ansible.cfg) = False
OS / ENVIRONMENT
STEPS TO REPRODUCE

Here is the working Playbook

- name: GRAB ZONE ID
  route53_zone:
    zone: "{{workshop_dns_zone}}"
  register: AWSINFO

- name: GRAB ROUTE53 INFORMATION
  route53_facts:
    type: A
    query: record_sets
    hosted_zone_id: "{{AWSINFO.zone_id}}"
    start_record_name: "student1.{{ec2_name_prefix|lower}}.{{workshop_dns_zone}}"
    max_items: 200
  register: record_sets

- name: PRINT OUT RECORD SETS
  debug:
    var: record_sets
EXPECTED RESULTS

I would expect I could return 200 items... not sure what the theoretically max is? If we can't return more than 100 I think we should error and say that.

ACTUAL RESULTS

Look at the MaxItems parameter below, it is still 100 despite the task above saying 200....

TASK [aws_dns : GRAB ROUTE53 INFORMATION] ******************************************************************************************************************************************************************
task path: /Users/sean/Documents/GitHub/linklight/provisioner/roles/aws_dns/tasks/teardown.yml:6
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: sean
<localhost> EXEC /bin/sh -c 'echo ~sean && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584 `" && echo ansible-tmp-1543874544.19-275131886232584="` echo /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584 `" ) && sleep 0'
Using module file /Library/Python/2.7/site-packages/ansible/modules/cloud/amazon/route53_facts.py
<localhost> PUT /Users/sean/.ansible/tmp/ansible-local-4199TKds5v/tmprtwf54 TO /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584/AnsiballZ_route53_facts.py
<localhost> EXEC /bin/sh -c 'chmod u+x /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584/ /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584/AnsiballZ_route53_facts.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584/AnsiballZ_route53_facts.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /Users/sean/.ansible/tmp/ansible-tmp-1543874544.19-275131886232584/ > /dev/null 2>&1 && sleep 0'
ok: [localhost] => {
    "IsTruncated": true,
    "MaxItems": "100",
    "NextRecordName": "rdu-student34.rhdemo.io.",
    "NextRecordType": "A",
    "ResourceRecordSets": [
        {
            "Name": "student1.junostesting.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "35.183.128.49"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "kbenson-napc.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "kbenson-napc.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "kbenson-test.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "kbenson-test.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "kbensonworkshop.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "kbensonworkshop.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "kev-bcbssc.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "kev-bcbssc.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "kevlowes.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "kevlowes.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "lee-student1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.144.170.139"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "lightbulb-dw.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "13.57.223.166"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "lightbulb.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.205.187.95"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "linklight-lee.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "student1-host1.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "172.18.2.41"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-rtr1.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.228.113.8"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-rtr2.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.84.19.200"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-tower.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "174.129.71.243"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-host1.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "172.18.4.131"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-rtr1.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.228.26.72"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-rtr2.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.235.170.219"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-tower.linklight-lee.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.224.70.70"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "kroger.linklight.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "kroger.linklight.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "nationwide.linklight.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "nationwide.linklight.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "docs.nationwide.linklight.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.227.36.105"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "med-redbox-ara.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "med-redbox-ara.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "melbourne-morning.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "melbourne-morning.rhdemo.io.s3-website.ap-southeast-2.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "melbourne-pm.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "melbourne-pm.rhdemo.io.s3-website.ap-southeast-2.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "melbourne.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "melbourne.rhdemo.io.s3-website.ap-southeast-2.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "napc-network.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "napc-network.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "student1-control.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.227.162.159"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-host1.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "172.18.2.178"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-rtr1.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.152.34.169"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-rtr2.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.91.67.167"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-control.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "184.72.124.46"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-host1.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "172.18.4.141"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-rtr1.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.207.60.42"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-rtr2.net-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.203.208.8"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "networking-demo-sean-gerald.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "networking-demo-sean-gerald.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "networking-forrester.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "networking-forrester.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "oa-ansible-ws.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.173.142.111"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-demo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.162.60.55"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "nginx.oa-demo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.89.141.82"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-demo2.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.146.180.180"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-demo4.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "13.57.39.146"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-linklight.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "oa-linklight.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "oa-linklight1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "oa-linklight1.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "oa-linklight2.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "oa-linklight2.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "oa-linklight3.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "oa-linklight3.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "oa-test.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.227.91.171"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-test2-student2-nginx.oa-test2.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.85.87.113"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-test6-rc1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.87.238.23"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oa-workshop1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.159.255.234"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "AliasTarget": {
                "DNSName": "elbrouter-ocp1-1031728793.us-east-1.elb.amazonaws.com.",
                "EvaluateTargetHealth": false,
                "HostedZoneId": "Z35SXDOTRQ7X7K"
            },
            "Name": "\\052.apps.ocp1.rhdemo.io.",
            "Type": "A"
        },
        {
            "AliasTarget": {
                "DNSName": "elbmaster-ocp1-313522863.us-east-1.elb.amazonaws.com.",
                "EvaluateTargetHealth": false,
                "HostedZoneId": "Z35SXDOTRQ7X7K"
            },
            "Name": "master.ocp1.rhdemo.io.",
            "Type": "A"
        },
        {
            "Name": "\\052.os_cluster1.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "18.205.117.30"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oscluster1.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.91.109.231"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "\\052.oscluster1.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.91.109.231"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oscluster2.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.152.120.36"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "\\052.oscluster2.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.152.120.36"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oscluster3.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.90.43.89"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "\\052.oscluster3.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.90.43.89"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "oscluster4.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.164.97.248"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "\\052.oscluster4.ocpdemo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.164.97.248"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "osp-test1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.207.164.114"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "partner-demo.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.210.64.68"
                }
            ],
            "TTL": 300,
            "Type": "A"
        },
        {
            "Name": "pnc-lightbulb.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.235.137.156"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "pnc-student1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.73.99.182"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "qos.rhdemo.io.s3-website-us-east-1.amazonaws.com"
                }
            ],
            "TTL": 3600,
            "Type": "CNAME"
        },
        {
            "Name": "student1-host1.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "172.18.2.169"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-rtr1.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.201.103.221"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-rtr2.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.201.200.38"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student1-tower.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.210.55.224"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-host1.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "172.18.4.104"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-rtr1.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.91.38.222"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-rtr2.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.196.120.20"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "student2-tower.qos.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.229.128.70"
                }
            ],
            "TTL": 60,
            "Type": "A"
        },
        {
            "Name": "rdu-student1.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.229.222.218"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student10.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.234.77.223"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student11.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.87.184.135"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student12.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.173.66.199"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student13.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.224.212.6"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student14.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.228.74.122"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student15.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.236.157.108"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student16.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.242.41.169"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student17.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.90.240.59"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student18.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.158.4.105"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student19.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.207.226.72"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student2.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "35.173.249.231"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student20.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.147.206.182"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student21.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "52.72.43.98"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student22.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.198.167.227"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student23.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.236.209.35"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student24.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.224.70.206"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student25.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.207.116.150"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student26.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.227.195.107"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student27.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "35.172.185.196"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student28.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.230.30.57"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student29.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.235.166.237"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student3.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.204.0.28"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student30.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.229.219.54"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student31.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.173.116.39"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student32.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "34.234.85.50"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        },
        {
            "Name": "rdu-student33.rhdemo.io.",
            "ResourceRecords": [
                {
                    "Value": "54.175.132.238"
                }
            ],
            "TTL": 3600,
            "Type": "A"
        }
    ],
    "ResponseMetadata": {
        "HTTPHeaders": {
            "content-length": "21830",
            "content-type": "text/xml",
            "date": "Mon, 03 Dec 2018 22:02:24 GMT",
            "vary": "Accept-Encoding",
            "x-amzn-requestid": "1dd09e0b-f747-11e8-8b62-c7b4382bf2e5"
        },
        "HTTPStatusCode": 200,
        "RequestId": "1dd09e0b-f747-11e8-8b62-c7b4382bf2e5",
        "RetryAttempts": 0
    },
    "changed": false,
    "invocation": {
        "module_args": {
            "aws_access_key": null,
            "aws_secret_key": null,
            "change_id": null,
            "delegation_set_id": null,
            "dns_name": null,
            "ec2_url": null,
            "health_check_id": null,
            "health_check_method": "list",
            "hosted_zone_id": "Z1TYCDVCKYS6QZ",
            "hosted_zone_method": "list",
            "max_items": "200",
            "next_marker": null,
            "profile": null,
            "query": "record_sets",
            "region": null,
            "resource_id": null,
            "security_token": null,
            "start_record_name": "student1.junostesting.rhdemo.io",
            "type": "A",
            "validate_certs": true
        }
    }
}

TASK [aws_dns : PRINT OUT RECORD SETS] *********************************************************************************************************************************************************************
task path: /Users/sean/Documents/GitHub/linklight/provisioner/roles/aws_dns/tasks/teardown.yml:15
ok: [localhost] => {
    "record_sets": {
        "IsTruncated": true,
        "MaxItems": "100",
        "NextRecordName": "rdu-student34.rhdemo.io.",
        "NextRecordType": "A",
        "ResourceRecordSets": [
            {
                "Name": "student1.junostesting.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "35.183.128.49"
                    }
                ],
                "TTL": 3600,
                "Type": "A"
            },
            {
                "Name": "melbourne-morning.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "melbourne-morning.rhdemo.io.s3-website.ap-southeast-2.amazonaws.com"
                    }
                ],
                "TTL": 3600,
                "Type": "CNAME"
            },
            {
                "Name": "melbourne-pm.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "melbourne-pm.rhdemo.io.s3-website.ap-southeast-2.amazonaws.com"
                    }
                ],
                "TTL": 3600,
                "Type": "CNAME"
            },
<<output removed for brevity>>
            {
                "Name": "rdu-student30.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "34.229.219.54"
                    }
                ],
                "TTL": 3600,
                "Type": "A"
            },
            {
                "Name": "rdu-student31.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "54.173.116.39"
                    }
                ],
                "TTL": 3600,
                "Type": "A"
            },
            {
                "Name": "rdu-student32.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "34.234.85.50"
                    }
                ],
                "TTL": 3600,
                "Type": "A"
            },
            {
                "Name": "rdu-student33.rhdemo.io.",
                "ResourceRecords": [
                    {
                        "Value": "54.175.132.238"
                    }
                ],
                "TTL": 3600,
                "Type": "A"
            }
        ],
        "ResponseMetadata": {
            "HTTPHeaders": {
                "content-length": "21830",
                "content-type": "text/xml",
                "date": "Mon, 03 Dec 2018 22:02:24 GMT",
                "vary": "Accept-Encoding",
                "x-amzn-requestid": "1dd09e0b-f747-11e8-8b62-c7b4382bf2e5"
            },
            "HTTPStatusCode": 200,
            "RequestId": "1dd09e0b-f747-11e8-8b62-c7b4382bf2e5",
            "RetryAttempts": 0
        },
        "changed": false,
        "failed": false
    }
}
@ansibot

This comment has been minimized.

Copy link
Contributor

@ansibot ansibot commented Dec 3, 2018

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibot

This comment has been minimized.

Copy link
Contributor

@ansibot ansibot commented Dec 3, 2018

@IPvSean, just so you are aware we have a dedicated Working Group for aws.
You can find other people interested in this in #ansible-aws on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

@ZuZuD

This comment has been minimized.

Copy link
Contributor

@ZuZuD ZuZuD commented Dec 5, 2018

It looks like AWS Route53 API endpoint is restricted MaxItems to 100:

In [15]: r = boto3.client('route53')  
In [16]: boto3.set_stream_logger(name='botocore')
In [17]: r.list_resource_record_sets(MaxItems='300',HostedZoneId='XXXXXXXX')
2018-12-05 18:45:10,243 botocore.hooks [DEBUG] Event before-parameter-build.route-53.ListResourceRecordSets: calling handler <function fix_route53_ids at 0x10be7c510>
DEBUG:botocore.hooks:Event before-parameter-build.route-53.ListResourceRecordSets: calling handler <function fix_route53_ids at 0x10be7c510>
2018-12-05 18:45:10,247 botocore.endpoint [DEBUG] Making request for OperationModel(name=ListResourceRecordSets) with params: {'url_path': '/2013-04-01/hostedzone/XXXXXXXXX/rrset', 'query_string': {'maxitems': '300'}, 'method': 'GET', 'headers': {'User-Agent': 'Boto3/1.8.5 Python/3.7.1 Darwin/16.7.0 Botocore/1.12.9'}, 'body': b'....
[...]
2018-12-05 18:45:10,608 botocore.parsers [DEBUG] Response body:
<ListResourceRecordSetsResponse xmlns="https://route53.amazonaws.com/doc/2013-04-01/"><ResourceRecordSets>[...]</ResourceRecordSets><IsTruncated>false</IsTruncated><MaxItems>100</MaxItems></ListResourceRecordSetsResponse>

I will add a comment in the max_items description to clarify that, but as we don't control this limit, there is not so much we can do.

@ansibot

This comment has been minimized.

Copy link
Contributor

@ansibot ansibot commented Dec 5, 2018

@IPvSean

This comment has been minimized.

Copy link
Contributor Author

@IPvSean IPvSean commented Dec 5, 2018

I will add a comment in the max_items description to clarify that, but as we don't control this limit, there is not so much we can do.

Couldn't we grab multiple requests? and add them to the same list basically obfuscating this limitation and letting an Ansible user override this AWS limitation?

At the very least we should error when you put something over 100... and link to the AWS documentation so people know its not Ansible... but imho that is a crutch that can be easily solved in some Python code.

@ZuZuD

This comment has been minimized.

Copy link
Contributor

@ZuZuD ZuZuD commented Dec 6, 2018

Couldn't we grab multiple requests? and add them to the same list basically obfuscating this limitation and letting an Ansible user override this AWS limitation?

There is a next_marker options for the module to grab multiple requests... Unfortunately the "record_sets" is not using the NextMarker token for the pagination but instead provide the next resource to query as the output is sorted (through NextRecordName and StartRecordName, and next query should use StartRecordName and StartRecordType) see here the API definition

At the very least we should error when you put something over 100... and link to the AWS documentation so people know its not Ansible... but imho that is a crutch that can be easily solved in some Python code.

This API limit of 100 is not documented by AWS and therefore might change. Moreover, even with a resource that support the NextMarker token (like hosted_zone) I am not sure we can leverage the playbook loop features to loop over all the markers with only two statements. (see the module description example)

That's probably better to get rid of the max_item option and to manage this transparently within the module. That would require to re-write the module.

@ewascent

This comment has been minimized.

Copy link

@ewascent ewascent commented Dec 11, 2018

Is there a way for users to set max_items in Ansible configuration to override the 300 item default, per module? This would at least allow users to make the limit apparent in our implementations.

@thecosmicfrog

This comment has been minimized.

Copy link

@thecosmicfrog thecosmicfrog commented Dec 17, 2018

+1 from me for this issue. This ended up being a frustrating problem, as I regularly work with Route 53 hosted zones with >100 record sets. I asked on Stack Overflow for assistance with this, and was provided this solution by a member of the community:
https://stackoverflow.com/a/53666067/2083329

@ansibot

This comment has been minimized.

Copy link
Contributor

@ansibot ansibot commented Jun 11, 2019

Files identified in the description:
None

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.