Berks install throws :gzip is not registered on Faraday::Response #1466
Comments
|
@dpetzel this is because berks is being called via bundler when you're calling |
|
We have many cookbooks, across many orgs, each of those is potentially using different Berkshelf versions. I appreciate that we can run berks directly, but ideally we can leverage the version the project has pinned to and avoid the "Hey on my machine I run a different version than you". This has been working for a rather long time as-is. Is there anything on my end that I can pin to preserve the previous working functionality? I'll give the ChefDK 0.9.0 a spin, but requiring folks upgrade to a package that release only this week feels harsh. |
|
@dpetzel the real problem is that you are using |
|
So I've upgraded to 0.9.0 and still get the same issue. I (along with a large number of folks I believe) were under the impression that I'm not trying to argue, but rather better understand, what is the proper way collaborate on cookbooks that having varying requirements around the version of chefspec, rubocop, foodcritic, and berkshelf. For those other tools we've been pretty ingrained in using Gemfiles to ensure a cookbook is tested against a consistent set of tooling, vs whatever happens to be on the developers workstation. Sorry if these are dumb or annoying questions, and I'm happy to ask them elsewhere if it would be more appropriate. Thanks |
|
@dpetzel if you're running Somebody at Chef is working on a blog post about testing cookbooks using ChefDK. It wasn't brought to my attention until yesterday that this anti-pattern of using Berkshelf from your Gemfile was so widespread throughout the community. It came to me as quite the surprise since it's been two years since we released the ChefDK - this is a sign to me that people weren't educated properly so hopefully this blog post will help. The answer to your question is that you simply don't put Berkshelf in your Gemfile. You instead you required your developers to have ChefDK on their machines and use the In the README of each project you should place a link to the ChefDK installer and the version that you know you support. Just don't use bundler and Gemfiles. Test Kitchen, Berkshelf, and a lot of the tools you already are including in a Gemfile are present inside the ChefDK. You don't need to re-install them with Bundler. If you have an extension to test kitchen, vagrant, or some other tool, that's when you use |
|
@reset Very much appreciate the explanation. As very pointed example of where this breaks down for us is rubocop. As they introduce new rules, are cookbooks go from passing (no violations) to failing because of new rules. Same story with foodcritic. We've been using Gemfiles to control those versions. For anyone else following allowing running I really do appreciate the time you've take to discuss this. I look forward to the blog post, but I dread that I need to modify all my CI jobs as I don't have a transitional period |
I found this error while debugging issues with #9. It seems that the Faraday httpclient has some sort of tuning set to transparently use gzip in the backaground. After removing this middleware I no longer saw the error. Not sure why nobody else saw this. Thoughts @reset?
|
I was able to get past this error using the following versions: |
|
@travis-altiscale Simply moving from |
|
@maoueh Yes, that's technically correct. @travis-altiscale's advice is a bit overkill. I would read my comments above, though, as installing from gem is an unsupported method for using Berkshelf. You should switch over to using ChefDK in your build pipelines and on your dev machines. |
|
@reset Sure, I read the whole thread before posting, I even started to reply but decided to not do so because my post was becoming a bit too big. As you sure know, ChefDK is a simple packaging of a Ruby runtime and gems set at predefined versions (and with pre-compiled version for native gems) creating an isolated environment immune to inter-dependencies clashing problems, just like this issue is about from my rough understanding. Those all-in-one-packages includes all runtime files of the language sadly duplicates a lot of stuff. I keep multi version of Vagrant because I develop plugin that I want to test in more than one version and I'm at around 4 Go in size just for it. In my opinion, the environment isolation problem should be solve by something else than a all-in-on-package, but that's another story. Like you said previously, it's been almost two years now that ChefDK is out, and still a lot of people are using the But I'm an advanced user. I restrict my current installed Ruby to one fully working with all Chef tooling. And I agree that "provisionning" another developer machine for cookbook development is cumbersome and it's where ChefDK really shine. I must admit that there is some resistance to change on my part also :D That's why I'm eager to read the article you are referring to and will probably give ChefDK another try. I'm sure there is good solutions to all problem I had with ChefDK, it's just a matter of taken time to solve them nicely ... or to wait enough to read a nice article leading the way. |
|
@reset @maoueh yeah I know I don't need to lock berkshelf-api-client, but since version 1.3.0 broke SSL verifying against the system CA bundle (needed since we use an internal CA for signing certificates and run our own berkshelf api server), I lock things down a little tighter so the pipeline doesn't break because of a surprise new version 😄 We also have no plans of switching to ChefDK because last time I tried it, it stomped all over RVM on my laptop. And on the pipeline side, we also try to put as little configuration on our jenkins slaves as possible so using bundler + Gemfile makes it job configuration instead. |
|
This is presenting itself to be very painful. We are finding a large number of cookbooks which are using varying ChefSpec versions. Some are still using the older version which relies on rspec2, and will require a significant lift to become rpsec3 compatible.
It appears ChefSpec relies on Berks as a Gem, and does not Is there really no way to pin something to previous versions so we can continue with the (now known to be unsupported) pattern thats been working for years? I'm feeling like I've been painted into a corner, while all along I thought I was protecting myself from this sort of thing by using the Gemfile to pin to versions we knew to work. I've got hundreds of cookbooks to contend with, and upgrading Berks in the Gemfile isn't going to fly, and it appears that simply "Use ChefDK, and not bundler" isn't feasible in the short term. I'd really appreciate and advice on how I can keep things working as they were, so we can have a chance to regroup and figure out path forward in doing things the "right" way. Thaks |
|
@dpetzel I recommended to you to bump to Berkshelf 4.x because bundler will resolve it just fine. I released it to help people in your situation while they migrated things over. |
|
@reset understood, but touching many hundreds of Gemfiles spread across dozens of teams is challenging. Additionally 4.x requires Ruby 2.0. Sadly we have developers who have no interest in adopting ChefDK, and given that we still run a large number of Chef 11 clients, there is value in still testing on Ruby 1.9. I'm really not trying to be a pain (despite how it might seem), but "fix forward" is a really painful solution for us, and if we could somehow get back to how things worked at the start of the week (even if we have to update Gemfiles to do so), would be really beneficial for us, vs forcing a mad dash to upgrade to Berks 4 |
|
@dpetzel Your last solution is to distribute a Gemfile.lock which contains the exact versions of the software that you need in each of the repositories |
|
Or a meta gem that is just a list of specific versions and have all of your cookbooks depend on that gem. This way you only have to change it in one place when you're ready to upgrade. |
|
Thanks guys, can I accomplish the same thing by pinning more strictly in my Gemfile vs distributing the lock?. I dug up a .lock from Sunday, which has this tree: |
|
@reset: What @dpetzel has said about foodcritic and rubocop really resonates with me. The linting rules are constantly being developed and modified with time. I should not be forced to update every single one of my cookbooks when the first guy on my development team decides to update his workstation. Pinning these gems in a Gemfile allows consistency at the cookbook level, not the workstation level, and is critical to maintain sanity among a diverse development team that's working on a lot of cookbooks. |
|
@icberg7 I'm not telling you that you need to stop using bundler for those things. You can if you insist. It's only Berkshelf (and other large apps like Vagrant) that you shouldn't be running with bundler. |
|
I came across the same problem recently and using Berkshelf 4.X fixed my problem. As such it's necessary to declare something like: This just feels less clean than simply stating the desired gem in the Gemfile for the thor tasks and using that. That being said we're all a little ignorant to bundler here so if there's a better way of calling ChefDK binaries from within a thor task that is called using a bundle (via a binstub) then we're happy to migrate to using the ChefDK binaries. |
|
@yoshiwaan berkshelf should be called by the |
|
I got exactly the same error and it stuck me for hours! I eventually fixed it by changing the versions of Here's part of my Gemfile if you're interested. Hopefully it helps~ |
|
Just one more quick note for anybody who hasn't followed along for this pretty long thread. Any advice that is being dropped here to modify your Gemfile and continue to use Berkshelf via Bundler is bad advice and should be considered a temporary solution to your problem. |
|
The easiest solution is to pin your |
|
@reset I hope I'm not derailing here but I guess it's in aid of moving to the ChefDK from bundler so... Like I said it's the same result when using /usr/bin/berks as when running So is the only option when shelling out to a Chef 'binary,' as it were, to call it with ::Bundle.with_clean_env? |
|
@yoshiwaan yes, the idea here is to spawn a new OS process without the inherited environment from your current process. There are various ways you can accomplish this with Ruby depending on which library you are using to shell out. |
The error is: ``` :gzip is not registered on Faraday::Response (Faraday::Error) ``` Upgrading to berkshelf >= 4.x solves this problem. See also here: berkshelf/berkshelf#1466
|
I'm somewhat (though not entirely) surprised that the gem install method is being warned against as not supported in favor of ChefDK. Indeed, I've had much better experience in general using the pre-packaged ChefDK rather than via RubyGems. I feel like this seems to be the eventual evolution of software in general. For example: bleeding edge users of build-from-source distros eventually move to prebuilt & packaged distros. Omnibus-packaged versions of software allow for less multifarious combinations of package versions, as many of the nodes in such a multidimensional test-matrix are usually broken in some way. In light of this however, I wonder what the recommended method of testing cookbooks against multiple versions of Chef / Ruby is nowadays? I ran into some issues like this one (also: #1464) when trying to enable testing of a cookbook on different versions of Ruby and Chef, using Travis-CI. Travis uses RVM and optionally multiple Are there any best-practices or suggestions for cookbook testing with Travis-CI these days? |
- Update berkshelf version to ~> 4.0 for Ruby > 1.9 to mitigate berkshelf/berkshelf#1466 - When Ruby is 1.9 lock down the following dependencies due to removal of 1.9 support - berkshelf - ridley - faraday - varia_model
|
@reset if i understand correct, you suggest more of a testing workflow like in the users-cookbook, |
|
@Rud5G yes, that's the correct approach! |
|
@reset Awesome |
|
Was starting a new cookbook today and happened to notice that If the plan is to not support that seems like it would be a good idea to take it out of the generator. |
|
@matschaffer thanks for the heads up, I've corrected that - #1485 |
|
Not to resurrect this issue from the dead, but for anyone else who finds this issue, I was able to work around it by updating bundler. When I bundled it with 1.6.2, I saw this error. After updating to 1.11.2, the error disappeared. I didn't update berkshelf or pin faraday - only updated bundler. My Gemfile.lock. |
I'm not entirely sure the issue is with Berkshelf or Faraday, but since its my entrypoint I'm starting here. When I attempt to run a berks install I get the Faraday exception below. To the best of my knowledge this started about two days and and nothing on our end has changed, but you know how that goes...
On my initial
chef exec bundle installit picked up faraday 0.9.2, which was new so I pinned to 0.9.0 based on the Berks gemspec. I get this same result regardless of which version I use.The text was updated successfully, but these errors were encountered: