Implement support for handling instance termination protection

[cristim]

Issue Type

• Feature Pull Request

Summary

This implements handling of the instance termination protection flag, requested as part of #267.

Current status:

• it compiles :-)
• hasn't been tested at all yet, didn't even try to run it against any infrastructure.
• tests had to be created for the entire region.scanInstances() function, which wasn't tested before, and are currently failing. I could use some help on getting them pass, since this is a bit harder to mock because of the pagination.
• It introduces lots of additional ec2.DescribeInstanceAttribute API calls, one per each instance, so it may need to be configurable for people who don't want to do all these calls. I think we should also do these calls only for on-demand instances running in ASGs, but this scope filtering is not implemented yet.

Code contribution checklist

1. The contribution fixes a single existing github issue, and it is linked
   to it.
2. The code is as simple as possible, readable and follows the idiomatic Go
   guidelines.
3. All new functionality is covered by automated test cases so the overall
   test coverage doesn't decrease.
4. No issues are reported when running make full-test.
5. Functionality not applicable to all users should be configurable.
6. Configurations should be exposed through Lambda function environment
   variables which are also passed as parameters to the
   CloudFormation
   and
   Terraform
   stacks defined as infrastructure code.
7. Global configurations set from the infrastructure stack level should also
   support per-group overrides using tags.
8. Tags names and expected values should be similar to the other existing
   configurations.
9. Both global and tag-based configuration mechanisms should be tested and
   proven to work using log output from various test runs.
10. The logs should be kept as clean as possible (use log levels as
    appropriate) and formatted consistently to the existing log output.
11. The documentation is updated to cover the new behavior, as well as the
    new configuration options for both stack parameters and tag overrides.
12. A code reviewer reproduced the problem and can confirm the code
    contribution actually resolves it.

[cristim]

I managed to write some tests and refactor the code a bit in the process, still needs to be tested against real AWS infrastructure.

[coveralls]

Coverage increased (+2.9%) to 83.774% when pulling 2866499 on feat/support-instance-termination-protection into fff582b on master.

[cristim]

I tested it against real infrastructure and it works pretty well.

[cristim]

Last chance to review this, I will merge it as it is on Friday unless I get any objections.

[lenucksi]

Most minor nits, the rest looks good to me.
Refactorings look nice, instanceMap makes things more readable.

Also, gofmt had minor complaints, that you might want to address.

Edit:

• thanks for review!
• I couldn't reproduce the gofmt issues

$ make fmt-check
ok      all files passed go fmt

[lenucksi]

You could consider adding 'unprotected' before 'on-demand'. The function below has a nice explaining name, so that could be enough to not need the more specific comment.
There is one more case of this throughout the PR.

[cristim]

good catch!
I deleted the comments, they should only explain why are we doing some harder to grasp things, not mention once again what are we're doing, which should be seen clearly in the code itself.

[cristim]

@lenucksi thanks for the review, I'll try to address these issues and I hope I can finally merge this tomorrow.