Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update documentation around checking in the Gemfile.lock #3372

Closed
indirect opened this issue Jul 20, 2017 · 17 comments
Closed

Update documentation around checking in the Gemfile.lock #3372

indirect opened this issue Jul 20, 2017 · 17 comments
Assignees
Labels

Comments

@indirect
Copy link
Member

@indirect indirect commented Jul 20, 2017

When Bundler first shipped, we gitignored the Gemfile.lock inside gems, following the advice given in this blog post.

Over time, however, it became clear that this practice forces the pain of broken dependencies onto new contributors, while leaving existing contributors potentially unaware of the problem. As a result, we have revised our guidance for gem authors to now recommend checking in the lock for gems. We have also removed Gemfile.lock from the gitignore file generated in new gems.

This is a tracking ticket for any further documentation changes needed to support gem developers in this new pattern of checking in Gemfile.lock to support new contributors, while continuing to run at least one CI build against the newest version of all dependencies, resolved from scratch at the time of the CI run.

@arbonap
Copy link

@arbonap arbonap commented Nov 22, 2017

I'd be happy to help out with documentation changes reflecting the new recommendation to check in the lock for gems.

Loading

bundlerbot referenced this issue in rubygems/bundler Feb 13, 2018
Update docs to reflect revised guidance to check in locks for gems

Thanks so much for the contribution!
To make reviewing this PR a bit easier, please fill out answers to the following questions.

### What was the end-user problem that led to this PR?

The problem was...
- Bundler stopped gitignoring gem locks. The change was merged in but we need to reflect that change in documentation.
### What was your diagnosis of the problem?

My diagnosis was...
- To add in documentation explaining why Bunder now does _not_ gitignore gem locks.
### What is your fix for the problem, implemented in this PR?

My fix...
- Update the `gemfile.lock` section of `bundle install` man pages as well as the `gemfile` man page.
### Why did you choose this fix out of the possible options?

I chose this fix because...
- This addresses open issue https://github.com/bundler/bundler/issues/5879
@colby-swandale
Copy link
Member

@colby-swandale colby-swandale commented Feb 20, 2018

This looks like it's been completed. Closing.

Loading

@mvz
Copy link
Contributor

@mvz mvz commented Feb 21, 2018

@colby-swandale I think this part has not yet been addressed:

while continuing to run at least one CI build against the newest version of all dependencies, resolved from scratch at the time of the CI run.

Loading

@colby-swandale
Copy link
Member

@colby-swandale colby-swandale commented Feb 23, 2018

@mvz You're right, thanks for pointing that out.

Loading

@rubymorillo
Copy link

@rubymorillo rubymorillo commented May 2, 2018

@indirect @colby-swandale I'm happy to take this one on. I need help clarifying this, though: where do we want this:

while continuing to run at least one CI build against the newest version of all dependencies, resolved from scratch at the time of the CI run.

to be called out? In the bundle-install.ronn man page?

Loading

@rubymorillo rubymorillo self-assigned this May 2, 2018
@segiddins
Copy link
Member

@segiddins segiddins commented May 3, 2018

Loading

@rubymorillo
Copy link

@rubymorillo rubymorillo commented May 4, 2018

@segiddins possibly. It's a how-to guide, though, and doesn't already make mention of CI. Here are the high-level sections:

  • Getting Started
  • Testing our gem
  • Using other gems
  • Testing a CLI
  • Crafting a CLI
  • Testing a generator
  • Writing a generator
  • Releasing the gem

At which point in that outline would it make most sense to call out CI builds? cc @indirect

Loading

@segiddins
Copy link
Member

@segiddins segiddins commented May 4, 2018

Probably within testing our gem?

Loading

colby-swandale referenced this issue in rubygems/bundler Oct 5, 2018
Update docs to reflect revised guidance to check in locks for gems

Thanks so much for the contribution!
To make reviewing this PR a bit easier, please fill out answers to the following questions.

### What was the end-user problem that led to this PR?

The problem was...
- Bundler stopped gitignoring gem locks. The change was merged in but we need to reflect that change in documentation.
### What was your diagnosis of the problem?

My diagnosis was...
- To add in documentation explaining why Bunder now does _not_ gitignore gem locks.
### What is your fix for the problem, implemented in this PR?

My fix...
- Update the `gemfile.lock` section of `bundle install` man pages as well as the `gemfile` man page.
### Why did you choose this fix out of the possible options?

I chose this fix because...
- This addresses open issue https://github.com/bundler/bundler/issues/5879

(cherry picked from commit 30f7b45)
colby-swandale referenced this issue in rubygems/bundler Oct 5, 2018
Update docs to reflect revised guidance to check in locks for gems

Thanks so much for the contribution!
To make reviewing this PR a bit easier, please fill out answers to the following questions.

### What was the end-user problem that led to this PR?

The problem was...
- Bundler stopped gitignoring gem locks. The change was merged in but we need to reflect that change in documentation.
### What was your diagnosis of the problem?

My diagnosis was...
- To add in documentation explaining why Bunder now does _not_ gitignore gem locks.
### What is your fix for the problem, implemented in this PR?

My fix...
- Update the `gemfile.lock` section of `bundle install` man pages as well as the `gemfile` man page.
### Why did you choose this fix out of the possible options?

I chose this fix because...
- This addresses open issue https://github.com/bundler/bundler/issues/5879

(cherry picked from commit 30f7b45)
ncoden referenced this issue in ncoden/foundation-rails Nov 16, 2018
Note: the Gemfile.lock lockfile should now be commited, even for gems.

See https://github.com/bundler/bundler/issues/5879
ncoden referenced this issue in ncoden/foundation-rails Nov 16, 2018
Note: the Gemfile.lock lockfile should now be commited, even for gems.

See https://github.com/bundler/bundler/issues/5879
netbsd-srcmastr referenced this issue in NetBSD/pkgsrc Dec 17, 2018
pkgsr change
* Remove @Prefix@ from ALTERNATIVES file.

## 1.17.2 (2018-12-11)

 - Add compatability for bundler merge with Ruby 2.6

## 1.17.1 (2018-10-25)

 - Convert `Pathname`s to `String`s before sorting them, fixing #6760 and #6758 ([#6761](rubygems/bundler#6761), @alexggordon)

## 1.17.0 (2018-10-25)

No new changes.

## 1.17.0.pre.2 (2018-10-13)

Features:

  - Configure Bundler home, cache, config and plugin directories with `BUNDLE_USER_HOME`, `BUNDLE_USER_CACHE`, `BUNDLE_USER_CONFIG` and `BUNDLE_USER_PLUGIN` env vars ([#4333](rubygems/bundler#4333), @gwerbin)
  - Add `--all` option to `bundle binstubs` that will generate an executable file for all gems with commands in the bundle
  - Add `bundle remove` command to remove gems from the Gemfile via the CLI
  - Improve checking file permissions and asking for `sudo` in Bundler when it doesn't need to
  - Add error message to `bundle add` to check adding duplicate gems to the Gemfile
  - When asking for `sudo`, Bundler will show a list of folders/files that require elevated permissions to write to.

The following new features are available but are not enabled by default. These are intended to be tested by users for the upcoming release of Bundler 2.

  - Improve deprecation warning message for `bundle show` command
  - Improve deprecation warning message for the `--force` option in `bundle install`

## 1.17.0.pre.1 (2018-09-24)

Features:

  - Check folder/file permissions of the Bundle home directory in the `bundle doctor` command ([#5786](rubygems/bundler#5786), @ajwann)
  - Remove compiled gem extensions when running `bundle clean` ([#5596](rubygems/bundler#5596), @akhramov)
  - Add `--paths` option to `bundle list` command ([#6172](rubygems/bundler#6172), @colby-swandale)
  - Add base error class to gems generated from `bundle gem` ([#6260](rubygems/bundler#6260), @christhekeele)
  - Correctly re-install gem extensions with a git source when running `bundle pristine` ([#6294](rubygems/bundler#6294), @wagenet)
  - Add config option to disable platform warnings ([#6124](rubygems/bundler#6124), @agrim123)
  - Add `--skip-install` option to `bundle add` command to add gems to the Gemfile without installation ([#6511](rubygems/bundler#6511), @agrim123)
  - Add `--only-explicit` option to `bundle outdated` to list only outdated gems in the Gemfile ([#5366](rubygems/bundler#5366), @peret)
  - Support adding multiple gems to the Gemfile with `bundle add` ([#6543](rubygems/bundler#6543), @agrim123)
  - Make registered plugin events easier to manage in the Plugin API (@jules2689)
  - Add new gem install hooks to the Plugin API (@jules2689)
  - Add `--optimistic` and `--strict` options to `bundle add` ([#6553](https://github.com/bundler/bundler/issues/6553), @agrim123)
  - Add `--without-group` and `--only-group` options to `bundle list` ([#6564](rubygems/bundler#6564), @agrim123)
  - Add `--gemfile` option to the `bundle exec` command ([#5924](rubygems/bundler#5924), @ankitkataria)

The following new features are available but are not enabled by default. These are intended to be tested by users for the upcoming release of Bundler 2.

  - Make `install --path` relative to the current working directory ([#2048](rubygems/bundler#2048), @igorbozato)
  - Auto-configure job count ([#5808](rubygems/bundler#5808), @segiddins)
  - Use the Gem Version Promoter for major gem updates ([#5993](rubygems/bundler#5993), @segiddins)
  - Add config option to add the Ruby scope to `bundle config path` when configured globally (@segiddins)

## 1.16.6 (2018-10-05)

Changes:

  - Add an error message when adding a gem with `bundle add` that's already in the bundle ([#6341](rubygems/bundler#6341), @agrim123)
  - Add Homepage, Source Code and Chanagelog URI metadata fields to the `bundle gem` gemspec template (@walf443)

Bugfixes:

  - Fix issue where updating a gem resulted in the gem's version being downgraded when `BUNDLE_ONLY_UPDATE_TO_NEWER_VERSIONS` was set ([#6529](rubygems/bundler#6529), @theflow)
  - Fix some rescue calls that don't specifiy error type (@utilum)
  - Fix an issue when the Lockfile would contain platform-specific gems that it didn't need ([#6491](rubygems/bundler#6491), @segiddins)
  - Improve handlding of adding new gems with only a single group to the Gemfile in `bundle add` (@agrim123)
  - Refactor check for OpenSSL in `bundle env` (@voxik)
  - Remove an unnecessary assignment in Metadata (@voxik)

Documentation:

  - Update docs to reflect revised guidance to check in Gemfile.lock into version control for gems ([#5879](https://github.com/bundler/bundler/issues/5879), @arbonap)
  - Add documentation for the `--all` flag in `bundle update` (@agrim123)
  - Update README to use `bundle add` in usage examples (@hdf1986)
@ianfixes
Copy link

@ianfixes ianfixes commented Nov 8, 2019

When Bundler first shipped, we gitignored the Gemfile.lock inside gems. [...] Over time, it became clear that this practice forces the pain of broken dependencies onto new contributors, while leaving existing contributors potentially unaware of the problem

This block of text is very difficult for me to understand. What is meant by "the pain of broken dependencies" (like, what activities cause this pain to be felt)? What are the "contributors" contributing to, an app or a library (gem)? And does "new contributors" mean "people who contribute new gems" or "people who make contributions to existing gems" (i.e. are we talking about 2 sets of developers and 1 gem or 2 sets of developers and 2 gems, or 2 sets of developers and an app and a gem, etc)?

One way I can interpret this is as follows:

When Bundler first shipped, we gitignored the Gemfile.lock inside gems. In theory, as a very popular article in a blog suggests, this allows people to use your library with versions of its dependencies that are different from the ones you used to develop the gem.

In practice, however, it became clear that not all versions of dependencies can work with a given gem. The dependencies worked when the code was initially contributed, so the original contributors may be unaware when they break. Instead, new contributors -- the people least likely to know how broken dependencies should be resolved -- will experience broken dependencies when they attempt to bundle install. As a result, we have revised our guidance for gem authors to now recommend checking in the lock for gems [SO THAT ______ HAPPENS INSTEAD].

Is that accurate?

The reason for my question is that I work in a DevOps role and am trying to work out what the best practices are for private gems within our organization -- there are no "new" or "existing" contributors, because we're all on the same team. And if I want any builds suffering from broken dependencies to fail as early and as loudly as possible, am I better off with or without Gemfile.lock checked into version control? What are the biggest risks of each option?

Loading

@mvz
Copy link
Contributor

@mvz mvz commented Nov 9, 2019

As I understand it, the tension is between:

  1. Not having a broken build just because a new version of a dependency is released.
  2. Learning about a new version of a dependency not working with your gem.

If you find 2. more important you should not check in your Gemfile.lock. For public gems, it makes sense to find 1. more important.

Ideally, you would have both: One build with a known good lockfile, and one with the most recent versions of the dependencies.

Loading

@indirect
Copy link
Member Author

@indirect indirect commented Nov 9, 2019

That’s a great summary. A checked in lock file is important so that new contributors and fresh checkouts of the repo aren’t broken. A build without a lock file is important to ensure that gem install is not broken. Ideally you’ll have both. 👍🏻

Loading

@ianfixes
Copy link

@ianfixes ianfixes commented Nov 10, 2019

OK. So the tradeoff is that by checking in Gemfile.lock, you ensure that fresh checkouts of the repo will always work -- by constricting the process of resolving dependencies.

On a practical level, this means that checking in Gemfile.lock requires you (the gem maintainer) to be proactive about testing the latest versions of your dependencies for compatibility.

I'd like to submit a paragraph of text that captures all these subtleties to replace what's currently listed at https://bundler.io/v2.0/guides/faq.html#committing-lockfiles-in-libraries

Where can I find that code for that page? And/or is there any process I need to worry about besides opening a pull request in that repo?

Loading

@deivid-rodriguez
Copy link
Member

@deivid-rodriguez deivid-rodriguez commented Nov 10, 2019

Note that proactively testing against newest versions of dependencies is very easy nowadays with dependabot. You can even configure it to create PRs in realtime after new releases of your dependencies!

Loading

@ianfixes
Copy link

@ianfixes ianfixes commented Nov 10, 2019

That's an interesting point but I want to stress that the thrust of my inquiry is how to evaluate what the best practices are (or should be) in a setting where the gems are private and/or the team is primarily static (i.e. no "new contributors").

I'm considering a few scenarios -- like I could check in the Gemfile.lock but delete it during certain CI or CD operations. Is there any interaction between having / not having a Gemfile.lock file and using / not using the --deployment flag with bundle install?

Loading

@indirect
Copy link
Member Author

@indirect indirect commented Nov 11, 2019

@ianfixes the best practices are the same. check in the Gemfile.lock so new contributors who get a new computer will have a working environment, and either keep a CI build that removes the lockfile or add Dependabot to your repo so you get PRs for every new version of your dependencies.

Loading

ianfixes referenced this issue in ianfixes/bundler-site Nov 18, 2019
Based on a discussion on GitHub:
https://github.com/bundler/bundler/issues/5879#issuecomment-551967102

This patch updates the documentation from saying "do this one thing" to explaining the pros and cons of committing a Gemfile.lock to source control.  This is important -- not only in light of the fact that the conventional wisdom has reversed itself in the past few years, but because what works for open source gems may not be best for privately-maintained gems.
@ianfixes
Copy link

@ianfixes ianfixes commented Nov 18, 2019

I've attempted to faithfully summarize everyone's comments and incorporate them into the existing documentation: rubygems/bundler-site#487

Loading

AlexWayfer referenced this issue in tycooon/memery Jan 24, 2020
@hsbt hsbt transferred this issue from rubygems/bundler Mar 14, 2020
@deivid-rodriguez
Copy link
Member

@deivid-rodriguez deivid-rodriguez commented Mar 14, 2020

Linked PR was merged in the documentation site, so I'll close this!

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants