SSM Waiter

[kotowick]

Is there going to be waiter names for SSM? It would be nice to be able to do a constant pull to see if the command ran successfully.

http://docs.aws.amazon.com/sdkforruby/api/Aws/SSM/Client.html#waiter_names-instance_method

[trevorrowe]

We are open to adding waiters to any/every service. The are no concrete plans for work on SSM waiters at the moment, but we can track this as a feature request. Pull requests are also welcome.

[awood45]

Added to feature request backlog.

[mullermp]

Reopening - deprecating usage of Feature Requests backlog markdown file.

[mullermp]

We're going to open an internal ticket with the SSM team and see if we can get any traction on adding waiters. This has sat around too long..

[goochi1]

Was there an update on this?

[mullermp]

Unfortunately no update here.. I have pushed several engineers on their team to implement them but I haven't gotten any traction on that request. Is this something you'd like to implement in the mean time? I can contribute those changes upstream so that all SDKs would get them.

[goochi1]

If you can link me to the code and how to contribute i can try and write one to submit. Was looking ealrier but couldnt see where it was

[mullermp]

As a small update, I've sent an email to managers of the SSM service with a link to this feature request as well as an internal ticket I had created.

If you'd like to take a crack at it, here are some waiter examples:
S3 waiter: https://github.com/aws/aws-sdk-ruby/blob/master/apis/s3/2006-03-01/waiters-2.json
DynamoDB waiter: https://github.com/aws/aws-sdk-ruby/blob/master/apis/dynamodb/2012-08-10/waiters-2.json

You basically define some JSON that says what operation to use and what the expected states (acceptors) are.

Here's how the generated code works:
https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/using-waiters.html

If you open a pull request, I can't merge it directly, but I can take the code and contribute it upstream.

[mullermp]

SSM team also referred me to this: https://docs.aws.amazon.com/systems-manager/latest/userguide/automation-aws-apis-calling.html Does this help with your use case?

[goochi1]

I am using the Python sdk on with boto3. should i be looking at a differnt repo or is teh ruby one the right one?

[mullermp]

Use that one - https://github.com/boto/boto3

Though waiters are shared across all SDKs so implementing would benefit all of them.

[goochi1]

Hey, I have finnaly got some time to look into this.

I looked at the ruby one but for SSM i think i need a bit more than the S3 ones. I am looking in the boto3 repo but cant see where the waiters would live. Probably being an idiot but can send me the link please?

[goochi1]

Heres what i have so far https://github.com/goochi1/aws-sdk-ruby/blob/ssm_waiter/apis/ssm/2014-11-06/waiters.json

[mullermp]

I'm not sure where they live in boto3, but any of the apis/*/*/waiters-2.json files in this repo will have various examples.

For example -
EC2 - https://github.com/aws/aws-sdk-ruby/blob/master/apis/ec2/2016-11-15/waiters-2.json
RDS - https://github.com/aws/aws-sdk-ruby/blob/master/apis/rds/2014-10-31/waiters-2.json
CF - https://github.com/aws/aws-sdk-ruby/blob/master/apis/cloudformation/2010-05-15/waiters-2.json

Are there any specific questions about waiters that I can help with? Is there a specific waiter you're looking to see implemented for your use case?

Regarding your implementation - on the surface it looks great :D thanks for starting this work. I will forward this to the SSM team to encourage them to make more! I can also help test these when they're ready.

[goochi1]

Hi Matt, I cant find any in the python one, it was more to see if i should add to that repo as thats my end goal to have the wiater in python.

As for specifics, I do. I have been looking at the S3 and EC2 waiter and saw that the operation need to be a of of the API commands i believe and for my personal use I need "GetCommandInvocation". (which i think related to the boto3 smm call 'get_command_invocation' which if you run too soon after 'send_command' it fails, hence the waiter. Have i got that logic right?

Anyway one question i have is do i need some kind of input in the wiater? or will it know from the API call?

Once we know this then we can have a go at testing. Would it be ok to book in a call to do this?
Steph :)

[mullermp]

All of the waiters are shared across SDKs - we have a set of internal-only packages that contain the JSON definitions of waiters, paginators, and resources for each service. Once they're merged upstream and released in a new version of SSM, it would be included in boto3 as well as ruby, etc.

I think your logic is correct. I have never used SSM personally, but I would assume that you'd call send_command and then invoke a waiter (such as the one you defined: :command_executed) that would "wait" for a 200, 301, or 403 status code and then return. GetCommandInvocation is the API, and various languages (such as python/ruby) will downcase/snakecase depending on the language convention. Without investigating/testing I'm unsure of those status codes however.

For waiter input, at least in the Ruby world, they're passed into the waiters:

ssm = Aws::SSM::Client.new

begin
  ssm.wait_until(:command_executed, document_name: 'mycooldocument', instance_ids:['i-12345678'])
  puts "ran command!"
rescue Aws::Waiters::Errors::WaiterFailed => error
  puts "failed waiting for command: #{error.message}"
end

[goochi1]

hey, so what i mean about the pass in is do you need to define what can be passed in into the json or does the API pickit up and know what to do? maybe we should test to just see

[mullermp]

In the JSON you are defining what API the waiter will use, how long to poll, and what type of response to look for. The waiter's source code is then built using that. Any params that the API use are passed into the waiter at runtime.

[goochi1]

Cool then this should work, how do we test?

[mullermp]

If you save the file in this path: apis/ssm/2014-11-06/waiters-2.json and then from the root directory (with Ruby installed) install all of the dependencies, build the gem, and run the interactive console:

bundle install
rake build:aws-sdk-ssm
gems/aws-sdk-resources/bin/aws-v3.rb -v

I should be able to help test this when I get available bandwidth.

[goochi1]

OK, i dont know anything about Ruby so i may just wait. Would you be free next week?

[mullermp]

I took a deeper look. Thanks for trying to do this, but I don't think it was right. I came up with this implementation instead. I've defined only one waiter, called CommandExecuted, that uses the "Status" value of the response instead of http response codes. I've tested to make sure this works and I'll get this merged upstream and talk with the SSM team to deploy it.

{
  "version": 2,
  "waiters": {
    "CommandExecuted": {
      "delay": 5,
      "operation": "GetCommandInvocation",
      "maxAttempts": 20,
      "acceptors": [
        {
          "expected": "Pending",
          "matcher": "path",
          "state": "retry",
          "argument": "Status"
        },
        {
          "expected": "InProgress",
          "matcher": "path",
          "state": "retry",
          "argument": "Status"
        },
        {
          "expected": "Delayed",
          "matcher": "path",
          "state": "retry",
          "argument": "Status"
        },
        {
          "expected": "Success",
          "matcher": "path",
          "state": "success",
          "argument": "Status"
        },
        {
          "expected": "Cancelled",
          "matcher": "path",
          "state": "failure",
          "argument": "Status"
        },
        {
          "expected": "TimedOut",
          "matcher": "path",
          "state": "failure",
          "argument": "Status"
        },
        {
          "expected": "Failed",
          "matcher": "path",
          "state": "failure",
          "argument": "Status"
        },
        {
          "expected": "Cancelling",
          "matcher": "path",
          "state": "failure",
          "argument": "Status"
        }
      ]
    }
  }
}

[mullermp]

@goochi1 @kotowick You'll be happy to hear that I got the CommandExecuted waiter merged and released by the SSM team. I'm sorry that the SSM team did not act sooner. This should be available in every SDK now.

https://github.com/aws/aws-sdk-ruby/blob/master/apis/ssm/2014-11-06/waiters-2.json

[goochi1]

@mullermp thanks for this! Does it mean it will be avalible in pythons boto3 now?