Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed - rds_instance: update instance to enable cloudwatch logs export does not work #52989

Open
wants to merge 1 commit into
base: devel
from

Conversation

Projects
None yet
3 participants
@sergey-trukhin
Copy link

sergey-trukhin commented Feb 26, 2019

SUMMARY

Fixes #51015

Fixes the issue when a user updates existing RDS instance with enabling CloudWatch logs export feature. If the instance was created without CloudWatch logs export, it's impossible to update the instance with the enable_cloudwatch_logs_exports parameter.

It happens because desired_option variable in module is list, but list doesn't have difference attribute. In addition, current_option and desired_option variables contain different data (i.e. current_option contains dict, but desired_option contains list)

ISSUE TYPE
  • Bugfix Pull Request
COMPONENT NAME

rds_instance

ADDITIONAL INFORMATION
- hosts: localhost
  tasks:
    - name: Create RDS instance
      rds_instance:
        state: present
        db_instance_identifier : "test-rds-instance"
        allocated_storage: 20
        db_instance_class: "db.t2.micro"
        db_name: "default_database"
        db_parameter_group_name: "default.postgres10"
        db_subnet_group_name: "default"
        engine: "postgres"
        engine_version: "10.6"
        master_user_password: "123456"
        master_username: "root"
        multi_az: no
        option_group_name: "default:postgres-10"
        region: "eu-west-1"
        vpc_security_group_ids: "sg-12345678"
        enable_cloudwatch_logs_exports:
          - postgresql
          - upgrade

Before:

ansible-playbook 2.8.0.dev0
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
No config file found; using defaults
Parsed localhost, inventory source with host_list plugin

PLAYBOOK: 1.yml ******************************************************************************************************************************************************************************************************************************1 plays in 1.yml

PLAY [localhost] *****************************************************************************************************************************************************************************************************************************META: ran handlers

TASK [Create RDS instance] *******************************************************************************************************************************************************************************************************************task path: /mnt/ansible/1.yml:45
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702 `" && echo ansible-tmp-1551187950.29-48457195400702="` echo /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702 `" ) && sleep 0'
Using module file /mnt/ansible/library/rds_instance.py
<localhost> PUT /root/.ansible/tmp/ansible-local-137QODWWg/tmpWm4xC7 TO /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/ /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/ > /dev/null 2>&1 && sleep 0'
The full traceback is:
Traceback (most recent call last):
  File "/root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py", line 113, in <module>
    _ansiballz_main()
  File "/root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py", line 105, in _ansiballz_main
    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
  File "/root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py", line 48, in invoke_module
    imp.load_module('__main__', mod, module, MOD_DESC)
  File "/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py", line 1163, in <module>
  File "/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py", line 1128, in main
  File "/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py", line 788, in get_parameters
  File "/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py", line 809, in get_options_with_changing_values
  File "/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py", line 884, in get_changing_options_with_inconsistent_keys
AttributeError: 'list' object has no attribute 'difference'

fatal: [localhost]: FAILED! => {
    "changed": false,
    "module_stderr": "Traceback (most recent call last):\n  File \"/root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py\", line 113, in <module>\n    _ansiballz_main()\n  File \"/root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py\", line 105, in _ansiballz_main\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\n  File \"/root/.ansible/tmp/ansible-tmp-1551187950.29-48457195400702/AnsiballZ_rds_instance.py\", line 48, in invoke_module\n    imp.load_module('__main__', mod, module, MOD_DESC)\n  File \"/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py\", line 1163, in <module>\n  File \"/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py\", line 1128, in main\n  File \"/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py\", line 788, in get_parameters\n  File \"/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py\", line 809, in get_options_with_changing_values\n  File \"/tmp/ansible_rds_instance_payload_MXfJWt/__main__.py\", line 884, in get_changing_options_with_inconsistent_keys\nAttributeError: 'list' object has no attribute 'difference'\n",
    "module_stdout": "",
    "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error",
    "rc": 1
}
        to retry, use: --limit @/mnt/ansible/1.retry

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************localhost                  : ok=0    changed=0    unreachable=0    failed=1

After:

ansible-playbook 2.8.0.dev0
  config file = None
  configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]
No config file found; using defaults
Parsed localhost, inventory source with host_list plugin

PLAYBOOK: 1.yml ******************************************************************************************************************************************************************************************************************************1 plays in 1.yml

PLAY [localhost] *****************************************************************************************************************************************************************************************************************************META: ran handlers

TASK [Create RDS instance] *******************************************************************************************************************************************************************************************************************task path: /mnt/ansible/1.yml:45
<localhost> ESTABLISH LOCAL CONNECTION FOR USER: root
<localhost> EXEC /bin/sh -c 'echo ~root && sleep 0'
<localhost> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143 `" && echo ansible-tmp-1551188002.44-193565308518143="` echo /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143 `" ) && sleep 0'
Using module file /mnt/ansible/library/rds_instance.py
<localhost> PUT /root/.ansible/tmp/ansible-local-171bjpggP/tmpYnZAIX TO /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143/AnsiballZ_rds_instance.py
<localhost> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143/ /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143/AnsiballZ_rds_instance.py && sleep 0'
<localhost> EXEC /bin/sh -c '/usr/bin/python /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143/AnsiballZ_rds_instance.py && sleep 0'
<localhost> EXEC /bin/sh -c 'rm -f -r /root/.ansible/tmp/ansible-tmp-1551188002.44-193565308518143/ > /dev/null 2>&1 && sleep 0'
changed: [localhost] => {
    "allocated_storage": 20,
    "auto_minor_version_upgrade": true,
    "availability_zone": "eu-west-1a",
    "backup_retention_period": 1,
    "ca_certificate_identifier": "rds-ca-2015",
    "changed": true,
    "copy_tags_to_snapshot": false,
    "db_instance_arn": "arn:aws:rds:eu-west-1:123456789123:db:test-rds-instance",
    "db_instance_class": "db.t2.micro",
    "db_instance_identifier": "test-rds-instance",
    "db_instance_port": 0,
    "db_instance_status": "available",
    "db_name": "default_database",
    "db_parameter_groups": [
        {
            "db_parameter_group_name": "default.postgres10",
            "parameter_apply_status": "in-sync"
        }
    ],
    "db_security_groups": [],
    "db_subnet_group": {
        "db_subnet_group_description": "default",
        "db_subnet_group_name": "default",
        "subnet_group_status": "Complete",
        "subnets": [
            {
                "subnet_availability_zone": {
                    "name": "eu-west-1a"
                },
                "subnet_identifier": "subnet-12345679",
                "subnet_status": "Active"
            },
            {
                "subnet_availability_zone": {
                    "name": "eu-west-1c"
                },
                "subnet_identifier": "subnet-12345678",
                "subnet_status": "Active"
            },
            {
                "subnet_availability_zone": {
                    "name": "eu-west-1b"
                },
                "subnet_identifier": "subnet-12345670",
                "subnet_status": "Active"
            }
        ],
        "vpc_id": "vpc-12345678"
    },
    "dbi_resource_id": "db-NDJKWOWIDMDNDYWUQ82CM313XI",
    "deletion_protection": false,
    "domain_memberships": [],
    "endpoint": {
        "address": "test-rds-instance.qwerty1234.eu-west-1.rds.amazonaws.com",
        "hosted_zone_id": "NXMWK2INDNAU",
        "port": 5432
    },
    "engine": "postgres",
    "engine_version": "10.6",
    "iam_database_authentication_enabled": false,
    "instance_create_time": "2019-02-26T18:29:00.153000+00:00",
    "invocation": {
        "module_args": {
            "allocated_storage": 20,
            "allow_major_version_upgrade": null,
            "apply_immediately": false,
            "auto_minor_version_upgrade": null,
            "availability_zone": null,
            "aws_access_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "aws_secret_key": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "backup_retention_period": null,
            "ca_certificate_identifier": null,
            "character_set_name": null,
            "copy_tags_to_snapshot": null,
            "creation_source": null,
            "db_cluster_identifier": null,
            "db_instance_class": "db.t2.micro",
            "db_instance_identifier": "test-rds-instance",
            "db_name": "default_database",
            "db_parameter_group_name": "default.postgres10",
            "db_security_groups": null,
            "db_snapshot_identifier": null,
            "db_subnet_group_name": "default",
            "domain": null,
            "domain_iam_role_name": null,
            "ec2_url": null,
            "enable_cloudwatch_logs_exports": [
                "postgresql",
                "upgrade"
            ],
            "enable_iam_database_authentication": null,
            "enable_performance_insights": null,
            "engine": "postgres",
            "engine_version": "10.6",
            "final_db_snapshot_identifier": null,
            "force_failover": null,
            "force_update_password": false,
            "iops": null,
            "kms_key_id": null,
            "license_model": null,
            "master_user_password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "master_username": "root",
            "monitoring_interval": null,
            "monitoring_role_arn": null,
            "multi_az": false,
            "new_db_instance_identifier": null,
            "option_group_name": "default:postgres-10",
            "performance_insights_kms_key_id": null,
            "performance_insights_retention_period": null,
            "port": null,
            "preferred_backup_window": null,
            "preferred_maintenance_window": null,
            "processor_features": null,
            "profile": null,
            "promotion_tier": null,
            "publicly_accessible": true,
            "purge_cloudwatch_logs_exports": true,
            "purge_tags": true,
            "read_replica": false,
            "region": "eu-west-1",
            "restore_time": null,
            "s3_bucket_name": null,
            "s3_ingestion_role_arn": null,
            "s3_prefix": null,
            "security_token": null,
            "skip_final_snapshot": false,
            "snapshot_identifier": null,
            "source_db_instance_identifier": null,
            "source_engine": null,
            "source_engine_version": null,
            "source_region": null,
            "state": "present",
            "storage_encrypted": null,
            "storage_type": null,
            "tags": null,
            "tde_credential_arn": null,
            "tde_credential_password": null,
            "timezone": null,
            "use_latest_restorable_time": null,
            "validate_certs": true,
            "vpc_security_group_ids": [
                "sg-12345678"
            ],
            "wait": true
        }
    },
    "latest_restorable_time": "2019-02-26T18:30:22+00:00",
    "license_model": "postgresql-license",
    "master_username": "root",
    "monitoring_interval": 0,
    "multi_az": false,
    "option_group_memberships": [
        {
            "option_group_name": "default:postgres-10",
            "status": "in-sync"
        }
    ],
    "pending_modified_values": {
        "pending_cloudwatch_logs_exports": {
            "log_types_to_enable": [
                "postgresql",
                "upgrade"
            ]
        }
    },
    "performance_insights_enabled": false,
    "preferred_backup_window": "22:38-23:08",
    "preferred_maintenance_window": "sat:03:04-sat:03:34",
    "publicly_accessible": true,
    "read_replica_db_instance_identifiers": [],
    "storage_encrypted": false,
    "storage_type": "gp2",
    "tags": {},
    "vpc_security_groups": [
        {
            "status": "active",
            "vpc_security_group_id": "sg-12345678"
        }
    ]
}
META: ran handlers
META: ran handlers

PLAY RECAP ***********************************************************************************************************************************************************************************************************************************localhost                  : ok=1    changed=1    unreachable=0    failed=0
@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Feb 26, 2019

@ansibot

This comment has been minimized.

Copy link
Contributor

ansibot commented Feb 26, 2019

@sergey-trukhin, just so you are aware we have a dedicated Working Group for aws.
You can find other people interested in this in #ansible-aws on Freenode IRC
For more information about communities, meetings and agendas see https://github.com/ansible/community

click here for bot help

@NeckBeardPrince

This comment has been minimized.

Copy link

NeckBeardPrince commented Mar 13, 2019

Any luck on getting this PR merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.