Determine best solution for clean merge to InSpec + ChefDK due to aws gem conflict

[adamleff]

As per the investigation in inspec/inspec#2258, we believe that merging inspec-aws into InSpec core may cause a Gem dependency conflict in the ChefDK project. Since InSpec is included in ChefDK, we need to ensure that InSpec's dependencies (and its soon-to-be-included projects, like inspec-aws) do not cause an inability to ship the ChefDK.

The outcome of this issue is a proposed solution (and a PR to address accordingly) on the dependency configuration of inspec-aws to make it a clean merge into InSpec and the ChefDK.

@jerryaldrichiii and @clintoncwolfe, please work together on this one.

[clintoncwolfe]

That is incorrect - 2258 does not conclude that merging inspec-aws to inspec core will cause a gem conflict on the aws-sdk gem.

2258 concludes that if inspec-aws moved to AWS SDK v3 (which we did not, see aborted #108 ) inspec-aws would conflict with audit cookbook, chef-dk, omnibus, and several other products.

When we do migrate to SDK v3, we'll need to coordinate across product lines. AFAIK, there is no cross-product issue tracking that.

We are under no pressure to move to SDK v3 at this time; v2 is API compatible.

[adamleff]

It's pretty easy to assume that there's still a remaining issue because the outcome of inspec/inspec#2258 was not clear. The last comment indicated that there may still be a conflict.

It's wonderful if there isn't, so let's use this issue to completely wrap this up, be extremely clear in a comment in this issue about what the recommendation is, any action items, and then close it out.

[clintoncwolfe]

Sounds good - I certainly would want to try cutting an integration release of inspec and chef-dk and checking for any issues, just for safety.

[jerryaldrichiii]

After some conversation we have decided that the way to ensure that there are no conflicts with ChefDK and the integration of inspec-aws into InSpec is to do the following:

1. Create branch on Train that has the AWS gems from inspec-aws in it's gemspec and push that branch
2. Create branch on InSpec that points to the Train branch and push that branch
3. Create branch on ChefDK that points to the new InSpec branch, do a bundle update, and then push
   that branch
4. Kick off an ad-hoc build of ChefDK in Jenkins and validate

I will take this as an action item and post the results here.

[clintoncwolfe]

@jerryaldrichiii, I've verified that only aws-sdk is required to be added to train. In inspec-aws, highline and nokogiri are cruft; unit and integration tests pass with them removed.

[jerryaldrichiii]

Great work @clintoncwolfe

I've got branches pushed that add the proper gem locations to test this with ChefDK. Now we just need to kick off an ad-hoc build.

https://github.com/chef/chef-dk/tree/ja/test-aws-gem
https://github.com/chef/inspec/tree/ja/test-aws-gem
https://github.com/chef/train/tree/ja/test-aws-gem

[jerryaldrichiii]

Testing complete! We were unable to get a successful Jenkins build for ChefDK, but we verified that ChefDK's Gemfile.lock contained the dependencies we expected and do not think that the integration of aws-sdk is the cause of the failure.

If we are wrong about this we will fix it in the future.