ec2 module: added support for attribute-based idempotency

[conradlee]

As discussed on the mailing list and in the irc channel, the aws modules would ideally provide better support for idempotent operations.

This pull request implements extended support for idempotently provisioning ec2 instances using the ec2 module. The "id" option is extended. If id is set to "tags", "group", "group_id", or "image", then new instances will be provisioned only if there are not already count=N such instances with the specified attribute. For example, if

  instance_tags: '{"db":"postgres"}'
  id: tag
  count: 5
  idempotence_attribute: instance_tags

then new instances will be provisioned only if there are not already five instances with the tag db: postgres key-value pair.

An example is included in the EXAMPLES string.

Note that this pull request should not break backwards compatibility. I have tested this module and it appears to work, but someone needs to make sure that upon exiting, the module correctly indicates whether anything has changed.

(A similar pull request was submitted here, but then closed.)

[conradlee]

Any thoughts on whether this is ready for a merge?

@jarv do you know whether this will play well with your pull request?

[jarv]

@conradlee Unfortunately it won't, I definitely want this feature though so I'm fine refactoring my pull or refactoring this one depending on which one is merged first.

[mpdehaan]

Question here, if the attribute is specified as tags, and you request count=5 and there are count=3, does it bring the count up to 2?

If so (A) that would be incredibly slick, and (B) I would love to have an example of that in the AWS section of the docs.

[conradlee]

@mpdehaan yes, that's the idea. To be more exact, let's say there are currently three ec2 instances with the tag '{"db":"postgres"}'. If, as in the example at the top of this pull request, you select idempotence_attribute: instance_tags and instance_tags: '{"db":"postgres"}' and count: 5, then two more ec2 instances are spun up.

I've added an example to the AWS documentation. Please look it over carefully and feel free to make changes.

[zbal]

That's a pretty sweet idea and it makes a bit more sense (to me) than the token based approach that last only as long as it (the token) can ...

[conradlee]

@mpdehaan any update on whether you want to accept the PR? As I mentioned above, I added an example to the AWS documentation as you requested.

[mpdehaan]

I generally find this hard for me to grok, I'd generally like to find a
more comprehensive solution to provisioning and am not sure this fits in
with the ansible way of doing things.

Stay tuned. Do need to think about this some.

On Sun, Jul 14, 2013 at 7:08 AM, Conrad Lee notifications@github.comwrote:

@mpdehaan https://github.com/mpdehaan any update on whether you want to
accept the PR? As I mentioned above, I added an example to the AWS
documentation as you requested.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/3299#issuecomment-20934866
.

[kirillzlds]

By the way, I'm using this feature for a while and I'm happy with that.

[mpdehaan]

I have decided I think we want to stick with just the native ID in amazon for this.

I do like the idea of the 'count' parameter saying I want N of these images, that would be neat stuff.

[teefax]

Changing the count parameter to increase/decrease the number of running instances and using the amazon ID as idempotency attribute won't work since the count parameter is tied to the parameters of the initial instance creation call:

msg: IdempotentParameterMismatch: Arguments on this idempotent request are inconsistent with arguments used in previous request(s).

So @conradlee's approach is possibly the only way to change the count parameter on the fly.

I've updated the patch to work with the latest devel code: teefax/ansible@772403b and it seems to work for instance creation -- I haven't tested instance deletion, though.