Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new connection plugin aws_ssm #49652

Merged
merged 53 commits into from Jan 11, 2020
Merged

Conversation

@psharkey
Copy link
Contributor

psharkey commented Dec 7, 2018

SUMMARY

This connection plugin allows Ansible to execute tasks on an EC2 instance via the Session Manager Plugin for the AWS CLI. This is an alternative to SSH connections which eliminates the need for jumpboxes and SSH keys. Similarly, for Windows this means the control machine does not need to have username/password credentials needed for WinRM to manage the remote EC2 instance.

Managed instances provides support for on-premise instances in hybrid environments.

ISSUE TYPE
  • Feature Pull Request
COMPONENT NAME

connection/aws_ssm

ADDITIONAL INFORMATION
Prerequisites
Sample Integration Test Playbook
---
#
# aws_ssm test playbook
#
- name: check plugin
  hosts: localhost
  gather_facts: False
  tags: syntax
  vars:
    plugin_path: '../connection_plugins' # Substitute your path to aws_ssm.py here
    ignore: 'E402,E501,W503,W504,E722,E741'
    plugins:
      - 'aws_ssm'
  tasks:
  - name:  ensure the plugin exists
    command: "ls -l {{ plugin_path }}/{{ item }}.py"
    changed_when: False
    with_items: "{{ plugins }}"
  - name: install pcodestyle
    pip:
      name: pycodestyle
    become: True
  - name: check plugin style
    command: pycodestyle {{ item }}.py --ignore={{ ignore }}
    changed_when: False
    args:
      chdir: "{{ plugin_path }}/"
    with_items: "{{ plugins }}"
  - name: check docs
    command: ansible-doc --type=connection --module-path={{ plugin_path }} aws_ssm
    changed_when: False
    with_items: "{{ plugins }}"

- name: check raw command
  hosts: linux
  gather_facts: False
  tags: raw
  vars:
    pwd_stdout: '/usr/bin'
  tasks:
  - name: raw
    raw: pwd
    register: raw
    changed_when: False
  - name: check raw output
    local_action:
      module: assert
      that:
        - "raw.stdout | trim == pwd_stdout"
        - "raw.stderr == ''"
      msg: "pwd stdout '{{ raw.stdout | trim }}' != '{{ pwd_stdout }}' or stderr not blank"

- name: check escalation
  hosts: linux
  gather_facts: False
  tags: escalation
  vars:
    whoami_become_root: 'root'
    whoami_become_user: 'ec2-user'
  tasks:
  - name: whoami become
    raw: whoami
    register: whoami_root
    changed_when: False
    become: True
  - name: check whoami output
    local_action:
      module: assert
      that:
        - "whoami_root.stdout | trim == whoami_become_root"
        - "whoami_root.stderr == ''"
      msg: "whoami stdout '{{ whoami_root.stdout | trim }}' != '{{ whoami_become_root }}' or stderr not blank"

  - name: whomai become_user
    raw: whoami
    register: whoami_user
    changed_when: False
    become_user: "{{ whoami_become_user }}"
    become: True
  - name: check whoami output
    local_action:
      module: assert
      that:
        - "whoami_user.stdout | trim == whoami_become_user"
        - "whoami_user.stderr == ''"
      msg: "whoami stdout '{{ whoami_user.stdout | trim }}' != '{{ whoami_become_user }}' or stderr not blank"

- name: install python-pip
  hosts: linux
  gather_facts: False
  tags:  python-pip
  tasks:
  - name: pip
    raw: sudo yum install -y python-pip
    tags: pip

- name: check the basics - linux
  hosts: linux
  gather_facts: False
  tags: smoke
  vars:
    content: 'Some test content'
    test_filename: 'test.txt'
    remote_file: "/tmp/{{ test_filename }}"
    local_file: "/tmp/{{ inventory_hostname }}.txt"
  tasks:
  - name: ping
    ping:
    tags: ping
  - name: facts
    setup:
    tags: facts
  - name: command
    command: 'uname -a'
    changed_when: False
    tags: command
  - name: copy
    copy:
      content: "{{ content }}"
      dest: "{{ remote_file }}"
    tags: copy
  - name: fetch
    fetch:
      src: "{{ remote_file }}"
      dest: "{{ local_file }}"
      flat: yes
      validate_checksum: false
    tags: fetch
  - name: check
    assert: 
      that:
        - "item == content"
      msg: "'{{ item }}' must match '{{ content }}'"
    with_file:
      - "{{ local_file }}"
    tags: check
  - name: cleanup remote
    file:
      path: "{{ remote_file }}"
      state: absent
    tags: cleanup
  - name: cleanup local
    local_action:
      module: file
      path: "{{ local_file }}"
      state: absent
    tags: cleanup

- name: check the basics - windows
  hosts: windows
  gather_facts: False
  tags: smoke
  vars:
    content: 'Some test content'
    test_filename: 'C:\test.txt'
    remote_file: "{{ test_filename }}"
    local_file: "/tmp/{{ inventory_hostname }}.txt"
  tasks:
  - name: win_ping
    win_ping:
    tags: ping
  - name: win_command
    win_command: 'whoami'
    changed_when: False
    tags: command
  - name: win_copy
    win_copy:
      content: "{{ content }}"
      dest: "{{ remote_file }}"
    tags: copy
  - name: win_cleanup remote
    win_file:
      path: "{{ remote_file }}"
      state: absent
    tags: cleanup

- name: check timeout
  hosts: linux
  gather_facts: False
  tags:
    - never
    - timeout
  vars:
    timeout_sleep: 30 # must be greater than ansible.cfg timeout (or cli -T, --timeout) value
  tasks:
  - name: expect fatal error due to timeout
    raw: sleep {{ timeout_sleep }}
Sample Inventory
test:
  hosts:
    linux:
      ansible_aws_ssm_instance_id:  i-063b4d30cb2b72ad8
    windows:
      ansible_aws_ssm_instance_id: i-03b2bbacfa4117f96
      ansible_shell_type: powershell
  children:
    us-east-1:
      hosts:
        linux:
      vars:
        ansible_aws_ssm_region: us-east-1
    us-west-2:
      hosts:
        windows:
      vars:
        ansible_aws_ssm_region: us-west-2
  vars:
    ansible_aws_ssm_bucket_name: your-bucket-name
    ansible_connection: aws_ssm

Example ad-hoc command using sample inventory (in hosts.yml):

$ ansible test -i ./hosts.yml -m raw -a pwd
linux | CHANGED | rc=0 >>
/usr/bin

windows | CHANGED | rc=0 >>

Path
----
C:\Windows\system32


$ 
Example managed Instance Dockerfile
#
# aws-ssm-agent w/activation
#
# Get activation: aws ssm create-activation --default-instance-name "docker-ubuntu" --iam-role "your-instance-role-name" --registration-limit 10 --region us-east-1
#
# Build: docker build -t managed --build-arg AWS_SSM_ACTIVATION_CODE=<YOUR_ACTIVATION_CODE> --build-arg AWS_SSM_ACTIVATION_ID=<YOUR_ACTIVATION_ID> .
#
# Run: docker run --rm -it --device /dev/ttyS0 managed
#
ARG BASE=ubuntu:bionic
FROM $BASE

# Install curl
RUN apt update -q && apt install -yq \
	    curl sudo \
  && apt-get clean \
  && rm -rf /var/lib/apt/lists/* \
  && rm -rf /tmp/*
  
ARG AWS_SSM_ACTIVATION_CODE
ARG AWS_SSM_ACTIVATION_ID
ARG AWS_REGION=us-east-1

RUN curl https://s3.amazonaws.com/ec2-downloads-windows/SSMAgent/latest/debian_amd64/amazon-ssm-agent.deb -o /tmp/amazon-ssm-agent.deb \
    && dpkg -i /tmp/amazon-ssm-agent.deb \
    && : "${AWS_SSM_ACTIVATION_ID:?Build argument needs to be set and non-empty.}" \
    && : "${AWS_SSM_ACTIVATION_CODE:?Build argument needs to be set and non-empty.}" \
    && amazon-ssm-agent -y -register -code $AWS_SSM_ACTIVATION_CODE -id $AWS_SSM_ACTIVATION_ID -region $AWS_REGION \
    && rm -rf /tmp/*

CMD ["amazon-ssm-agent", "start"]
@ansibot

This comment was marked as resolved.

Copy link
Contributor

ansibot commented Dec 8, 2018

The test ansible-test sanity --test docs-build [explain] failed with 6 errors:

docs/docsite/rst/plugins/connection.rst:12:0: undefined-label: undefined label: local_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:12:0: undefined-label: undefined label: paramiko_ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:12:0: undefined-label: undefined label: ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:46:0: undefined-label: undefined label: paramiko_ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:46:0: undefined-label: undefined label: ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:50:0: undefined-label: undefined label: ssh_connection (if the link has no caption the label must precede a section header)

click here for bot help

@ansibot

This comment was marked as resolved.

Copy link
Contributor

ansibot commented Dec 8, 2018

The test ansible-test sanity --test docs-build [explain] failed with 6 errors:

docs/docsite/rst/plugins/connection.rst:12:0: undefined-label: undefined label: local_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:12:0: undefined-label: undefined label: paramiko_ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:12:0: undefined-label: undefined label: ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:46:0: undefined-label: undefined label: paramiko_ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:46:0: undefined-label: undefined label: ssh_connection (if the link has no caption the label must precede a section header)
docs/docsite/rst/plugins/connection.rst:50:0: undefined-label: undefined label: ssh_connection (if the link has no caption the label must precede a section header)

click here for bot help

@ansibot ansibot added core_review and removed needs_revision labels Dec 8, 2018
@psharkey psharkey force-pushed the psharkey:aws_ssm-connection-plugin branch 2 times, most recently Dec 10, 2018
@samdoran samdoran removed the needs_triage label Dec 11, 2018
@ansibot ansibot added the stale_ci label Dec 22, 2018
@ansibot

This comment was marked as resolved.

Copy link
Contributor

ansibot commented Dec 26, 2018

@psharkey this PR contains the following merge commits:

Please rebase your branch to remove these commits.

click here for bot help

@ansibot

This comment was marked as resolved.

Copy link
Contributor

ansibot commented Dec 26, 2018

The test ansible-test sanity --test pylint [explain] failed with 1 error:

lib/ansible/plugins/connection/aws_ssm.py:222:0: anomalous-backslash-in-string Anomalous backslash in string: '\d'. String constant might be missing an r prefix.

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/plugins/connection/aws_ssm.py:222:22: W605 invalid escape sequence '\d'

click here for bot help

@psharkey psharkey force-pushed the psharkey:aws_ssm-connection-plugin branch to c20d68e Dec 26, 2018
@ansibot ansibot added the stale_ci label Jan 6, 2019
@psharkey psharkey force-pushed the psharkey:aws_ssm-connection-plugin branch from 4649409 to 9cb0559 Jan 7, 2020
@jillr jillr requested a review from s-hertel Jan 8, 2020
@ansibot ansibot added the core_review label Jan 8, 2020
@jillr

This comment has been minimized.

Copy link
Contributor

jillr commented Jan 9, 2020

Hi @hanumantharaomvl, got your irc message. The #ansible-aws channel has most of the people who work on the aws modules so that's usually the best place to ask questions or request additional reviews if you ever need it.
https://github.com/ansible/ansibullbot/blob/master/ISSUE_HELP.md#labels

Copy link
Contributor

s-hertel left a comment

Besides a few suggestions for the tests this looks good to merge to me.

@Console32

This comment has been minimized.

Copy link

Console32 commented Jan 10, 2020

First thanks all for the work on this :)

I do still hit some errors when using this with windows 2012 due to OSC Characters: (They are used by powershell to set the window title)

failed to transfer file to /Users/rschn/.ansible/tmp/ansible-local-860678nqt3g7g/tmp8umt4imi \u001b]0;Windows PowerShell\u0007\u001b]0;Administrator: Windows PowerShell\u0007C:\Windows\TEMP\ansible-tmp-1578641908.257526-107021574924487\AnsiballZ_win_file.ps1:

I fixed this by adding the following two lines to the _filter_ansi method:

       if self.is_windows:
            osc_filter = re.compile(r'\x1b\][^\x07]*\x07') # <--
            line = osc_filter.sub('', line)                # <--
            ansi_filter = re.compile(r'(\x9B|\x1B\[)[0-?]*[ -/]*[@-~]')
            line = ansi_filter.sub('', line)

Can you include this?

@ansibot ansibot removed the core_review label Jan 10, 2020
@hanumantharaomvl hanumantharaomvl force-pushed the psharkey:aws_ssm-connection-plugin branch from 85f0b15 to d46de14 Jan 10, 2020
@jctanner

This comment has been minimized.

Copy link
Member

jctanner commented Jan 10, 2020

!waffling core_review

@psharkey

This comment has been minimized.

Copy link
Contributor Author

psharkey commented Jan 10, 2020

@Console32 the update for Windows 2012 is in d46de14.

@s-hertel

This comment has been minimized.

Copy link
Contributor

s-hertel commented Jan 10, 2020

bot_broken

@ansibot ansibot added the bot_broken label Jan 10, 2020
@jillr
jillr approved these changes Jan 11, 2020
Copy link
Contributor

jillr left a comment

Thanks again everyone for all the work developing and testing this new plugin - looks good to me, let's shipit!

@jillr jillr merged commit f8fb391 into ansible:devel Jan 11, 2020
1 check passed
1 check passed
Shippable Run 155540 status is SUCCESS.
Details
@hanumantharaomvl

This comment has been minimized.

Copy link
Contributor

hanumantharaomvl commented Jan 11, 2020

Hi @jillr Thank you and appreciate all your time reviewing the PR and for merging it.

@hanumantharaomvl

This comment has been minimized.

Copy link
Contributor

hanumantharaomvl commented Jan 11, 2020

HI @kapilt , @mattclay , @jamescassell , @jborean93 , @s-hertel , @jillr, Thank you for all your time for reviewing the PR, giving great insights and suggestions for improvement and helping the PR to get merged. Appreciate.

@psharkey

This comment has been minimized.

Copy link
Contributor Author

psharkey commented Jan 11, 2020

Ditto to @hanumantharaomvl 's comments. Thank you all!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.