Update documentation around checking in the Gemfile.lock

[indirect]

When Bundler first shipped, we gitignored the Gemfile.lock inside gems, following the advice given in this blog post.

Over time, however, it became clear that this practice forces the pain of broken dependencies onto new contributors, while leaving existing contributors potentially unaware of the problem. As a result, we have revised our guidance for gem authors to now recommend checking in the lock for gems. We have also removed Gemfile.lock from the gitignore file generated in new gems.

This is a tracking ticket for any further documentation changes needed to support gem developers in this new pattern of checking in Gemfile.lock to support new contributors, while continuing to run at least one CI build against the newest version of all dependencies, resolved from scratch at the time of the CI run.

[arbonap]

I'd be happy to help out with documentation changes reflecting the new recommendation to check in the lock for gems.

[colby-swandale]

This looks like it's been completed. Closing.

[mvz]

@colby-swandale I think this part has not yet been addressed:

while continuing to run at least one CI build against the newest version of all dependencies, resolved from scratch at the time of the CI run.

[colby-swandale]

@mvz You're right, thanks for pointing that out.

[rubymorillo]

@indirect @colby-swandale I'm happy to take this one on. I need help clarifying this, though: where do we want this:

while continuing to run at least one CI build against the newest version of all dependencies, resolved from scratch at the time of the CI run.

to be called out? In the bundle-install.ronn man page?

[segiddins]

Maybe in https://bundler.io/v1.16/guides/creating_gem.html ?

[rubymorillo]

@segiddins possibly. It's a how-to guide, though, and doesn't already make mention of CI. Here are the high-level sections:

• Getting Started
• Testing our gem
• Using other gems
• Testing a CLI
• Crafting a CLI
• Testing a generator
• Writing a generator
• Releasing the gem

At which point in that outline would it make most sense to call out CI builds? cc @indirect

[segiddins]

Probably within testing our gem?

[ianfixes]

When Bundler first shipped, we gitignored the Gemfile.lock inside gems. [...] Over time, it became clear that this practice forces the pain of broken dependencies onto new contributors, while leaving existing contributors potentially unaware of the problem

This block of text is very difficult for me to understand. What is meant by "the pain of broken dependencies" (like, what activities cause this pain to be felt)? What are the "contributors" contributing to, an app or a library (gem)? And does "new contributors" mean "people who contribute new gems" or "people who make contributions to existing gems" (i.e. are we talking about 2 sets of developers and 1 gem or 2 sets of developers and 2 gems, or 2 sets of developers and an app and a gem, etc)?

One way I can interpret this is as follows:

When Bundler first shipped, we gitignored the Gemfile.lock inside gems. In theory, as a very popular article in a blog suggests, this allows people to use your library with versions of its dependencies that are different from the ones you used to develop the gem.

In practice, however, it became clear that not all versions of dependencies can work with a given gem. The dependencies worked when the code was initially contributed, so the original contributors may be unaware when they break. Instead, new contributors -- the people least likely to know how broken dependencies should be resolved -- will experience broken dependencies when they attempt to bundle install. As a result, we have revised our guidance for gem authors to now recommend checking in the lock for gems [SO THAT ______ HAPPENS INSTEAD].

Is that accurate?

The reason for my question is that I work in a DevOps role and am trying to work out what the best practices are for private gems within our organization -- there are no "new" or "existing" contributors, because we're all on the same team. And if I want any builds suffering from broken dependencies to fail as early and as loudly as possible, am I better off with or without Gemfile.lock checked into version control? What are the biggest risks of each option?

[mvz]

As I understand it, the tension is between:

1. Not having a broken build just because a new version of a dependency is released.
2. Learning about a new version of a dependency not working with your gem.

If you find 2. more important you should not check in your Gemfile.lock. For public gems, it makes sense to find 1. more important.

Ideally, you would have both: One build with a known good lockfile, and one with the most recent versions of the dependencies.

[indirect]

That’s a great summary. A checked in lock file is important so that new contributors and fresh checkouts of the repo aren’t broken. A build without a lock file is important to ensure that gem install is not broken. Ideally you’ll have both. 👍🏻

[ianfixes]

OK. So the tradeoff is that by checking in Gemfile.lock, you ensure that fresh checkouts of the repo will always work -- by constricting the process of resolving dependencies.

On a practical level, this means that checking in Gemfile.lock requires you (the gem maintainer) to be proactive about testing the latest versions of your dependencies for compatibility.

I'd like to submit a paragraph of text that captures all these subtleties to replace what's currently listed at https://bundler.io/v2.0/guides/faq.html#committing-lockfiles-in-libraries

Where can I find that code for that page? And/or is there any process I need to worry about besides opening a pull request in that repo?

[deivid-rodriguez]

Note that proactively testing against newest versions of dependencies is very easy nowadays with dependabot. You can even configure it to create PRs in realtime after new releases of your dependencies!

[ianfixes]

That's an interesting point but I want to stress that the thrust of my inquiry is how to evaluate what the best practices are (or should be) in a setting where the gems are private and/or the team is primarily static (i.e. no "new contributors").

I'm considering a few scenarios -- like I could check in the Gemfile.lock but delete it during certain CI or CD operations. Is there any interaction between having / not having a Gemfile.lock file and using / not using the --deployment flag with bundle install?

[indirect]

@ianfixes the best practices are the same. check in the Gemfile.lock so new contributors who get a new computer will have a working environment, and either keep a CI build that removes the lockfile or add Dependabot to your repo so you get PRs for every new version of your dependencies.

[ianfixes]

I've attempted to faithfully summarize everyone's comments and incorporate them into the existing documentation: rubygems/bundler-site#487

[deivid-rodriguez]

Linked PR was merged in the documentation site, so I'll close this!