AMI 4.x support

[robert2d]

• additional info added
• S3DistCp defaults to new JAR location for AMI >= 4.x

[robert2d]

Hey @fblundun, I made a couple of changes to your code and added a few specs for the new features. but your commits are still in there ;)

[fblundun]

Thanks for the release @robert2d , it's working well!

One more question: for AMI 3.x, we had to do some additional Hadoop configuration for a particular input format. This is how we did it using Elasticity's HadoopBootstrapAction.

But in 4.x, predefined bootstrap actions are replaced by configurations. It looks like we should start using the new Elasticity::Application class. Do you have any idea how to translate our old bootstrap logic into an Application configuration?

Many thanks!

[fblundun]

Update: I didn't see a way to set configurations in the Elasticity API so I ended up putting them directly into the payload by monkey patching the AwsRequestV4 class here.

[robert2d]

Yea looks like a new "feature" that needs to added. I'll create an issue for now

[robert2d]

On the subject of Applications you can see them in the readme now here