Diplomat gem fails on Chef 16.9.29+

[Sapr0]

🗣️ Foreword

Thank for taking the time to fill this bug report fully. Without it we may not be able to fix the bug, and the issue may be closed without resolution.

👻 Brief Description

Cookbooks fails to load diplomat gem

🥞 Cookbook version

4.3.1

👩‍🍳 Chef-Infra Version

Chef Infra Client: 16.9.29

🎩 Platform details

Amazon Linux AMI release 2018.03

Steps To Reproduce

Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

🚓 Expected behavior

I expect the cookbook to run without errors

➕ Additional context

According to Chef release notes https://docs.chef.io/release_notes_client/#whats-new-in-16929 most notable change was

Duplicate gems in our packaging have been removed to further shrink the package sizes and improve load time.

Going down the rabbit hole, and comparing embedded gem list between 16.9.20 and 16.9.29 the faraday gem was bumped to 1.3.0 among others.

However, according to diplomat gemspec: https://github.com/WeAreFarmGeek/diplomat/blob/master/diplomat.gemspec#L28
They require faraday lower then 1.1.0, to support older versions of ruby.

So while this seems like a diplomat gem issue, it's really a cookbook issue that decided to use this gem.
According to chef stacktrace:

>>>> Caused by Gem::ConflictError: Unable to activate diplomat-2.4.4, because faraday-1.3.0 conflicts with faraday (>= 0.9, < 1.1.0)
/opt/chef/embedded/lib/ruby/2.7.0/rubygems/specification.rb:2247:in `raise_if_conflicts'
/opt/chef/embedded/lib/ruby/2.7.0/rubygems/specification.rb:1369:in `activate'
/opt/chef/embedded/lib/ruby/2.7.0/rubygems.rb:217:in `rescue in try_activate'
/opt/chef/embedded/lib/ruby/2.7.0/rubygems.rb:210:in `try_activate'
/opt/chef/embedded/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:152:in `rescue in require'
/opt/chef/embedded/lib/ruby/2.7.0/rubygems/core_ext/kernel_require.rb:147:in `require'
/etc/chef/local-mode-cache/cache/cookbooks/consul/libraries/consul_policy.rb:89:in `configure_diplomat'
/etc/chef/local-mode-cache/cache/cookbooks/consul/libraries/consul_policy.rb:62:in `action_create'

If we try to forcefully remove faraday 1.3.0:

/opt/chef/embedded/bin/gem uninstall faraday -v 1.3.0
Successfully uninstalled faraday-1.3.0

Then we break chef entirely:

chef-client -v
Traceback (most recent call last):
	3: from /usr/bin/chef-client:45:in `<main>'
	2: from /opt/chef/embedded/lib/ruby/2.7.0/rubygems/core_ext/kernel_gem.rb:62:in `gem'
	1: from /opt/chef/embedded/lib/ruby/2.7.0/rubygems/dependency.rb:323:in `to_spec'
/opt/chef/embedded/lib/ruby/2.7.0/rubygems/dependency.rb:313:in `to_specs': Could not find 'faraday' (= 1.3.0) - did find: [faraday-1.0.1] (Gem::MissingSpecVersionError)
Checked in 'GEM_PATH=/root/.gem/ruby/2.7.0:/opt/chef/embedded/lib/ruby/gems/2.7.0', execute `gem env` for more information

So now you need to put in your metadata.rb that you don't support Chef higher then 16.9.29, until you convince the diplomat gem to fix the faraday requirement or chose a different implementation then diplomat gem

[ramereth]

This is really an upstream issue. I'm happy to see that you've opened up an issue and PR to get it fixed there.

This does bring up the fact that we don't test this at all in the current kitchen tests. Do you think you could add a suite which tests it so we can verify this is fixed upstream when it's released?

[pierresouchay]

A new release of diplomat 2.5.0 has been created (which breaks the pre ruby 2.5 compatibility), hope it will fix the issue thanks to @Sapr0

[Sapr0]

@ramereth i looked at one of the runs: https://github.com/sous-chefs/consul/runs/1590571470?check_suite_focus=true

It seems the kitchen tests are on chef 17.0.1
Not sure how you want to test a combination of OS and chef version, it can easily explode into a cartesian product

[Sapr0]

I've re-run the cookbook and it now downloads the newest diplomat gem and everything is working as expected.