New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use credentials from bundler config #40

Open
wants to merge 4 commits into
base: master
from

Conversation

Projects
None yet
2 participants
@burke
Copy link

burke commented Jan 16, 2019

login:password can be specified in bundler configs.

This behaviour feels a little more in line with how bundler itself operates.

EDIT: Oh, well, this works for bundix -l but I still get a 401 in nix-build, which is puzzling, since it should have downloaded the file already.

Use credentials from bundler config:
login:password can be specified in bundler configs:

https://bundler.io/v1.15/man/bundle-config.1.html#CREDENTIALS-FOR-GEM-SOURCES

If these are specified, and the URL did not specify explicit auth,
use the authentication information from the URL.
Show resolved Hide resolved lib/bundix/source.rb Outdated
@alyssais
Copy link
Collaborator

alyssais left a comment

I'd prefer that we require "bundler" and use it to pull the data we need to fetch. Reimplementing this ourselves is brittle and won't capture the intricacies of how Bundler works. For example, Bundler allows the config file location to be customised, but with this patch authentication would fail in that case.

@burke

This comment has been minimized.

Copy link
Author

burke commented Jan 17, 2019

Ok great, that's probably a simpler implementation anyway. I'll revise this today. 👍

@burke

This comment has been minimized.

Copy link
Author

burke commented Jan 17, 2019

I've revised this PR to use Bundler::Settings directly, and to gracefully handle the case where bundler isn't available, for whatever reason.

With an incorrect token, then a correct one:

screen shot 2019-01-17 at 2 31 22 pm

@burke

This comment has been minimized.

Copy link
Author

burke commented Jan 17, 2019

Though, I'm confused by this:

building '/nix/store/v408l83vns1548cwnvqa0m11xfr65zl1-gemfile-and-lockfile.drv'...
building '/nix/store/a22rs6cn6pyiql2rciygfmhrzs71vm8g-shopify-cops-0.0.11.gem.drv'...

trying https://packages.shopify.io/shopify/gems/gems/shopify-cops-0.0.11.gem
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 401 Unauthorized
error: cannot download shopify-cops-0.0.11.gem from any mirror
builder for '/nix/store/a22rs6cn6pyiql2rciygfmhrzs71vm8g-shopify-cops-0.0.11.gem.drv' failed with exit code 1
cannot build derivation '/nix/store/rdzhzhmd716pmk55d67ncf4rcrn7m6wv-ruby2.5.3-shopify-cops-0.0.11.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/2dk6sidr07wam9v99wl83x6i6y8vmq01-dev.drv': 1 dependencies couldn't be built
error: build of '/nix/store/2dk6sidr07wam9v99wl83x6i6y8vmq01-dev.drv' failed

Isn't the purpose of this code that I've modified to prefetch the *.gem files so that nix won't have to do it later? Am I misunderstanding? Is there other code to teach about bundler credentials, or is it somehow mis-predicting the derivation hash?

@alyssais

This comment has been minimized.

Copy link
Collaborator

alyssais commented Jan 19, 2019

Have you tried deleting gemset.nix? There’s something weird that happens when it already exists that I haven’t had the time to track down yet.

@burke

This comment has been minimized.

Copy link
Author

burke commented Jan 22, 2019

The issue was that Bundix inserts into the nix-store entries like:
q3vq532pnpvq19sca3ccgi5yabi71y2x-https_rubygems_org_gems_rubocop-0_60_0_gem, while bundlerEnv seems to expect 06sp9w0i042db7y8h1gx7vwkyiw4cmpx-rubocop-0.60.0.gem.

This would cause the gems to be downloaded twice: once by bundix and then again by bundlerEnv. Since I haven't provided bundlerEnv with a way to retrieve credentials, that failed.

I've amended this PR to write files into the store at the correct paths as expected by bundlerEnv.

I also removed some dead code: I'm happy to move this to its own PR if it's easier to review that way but it seemed like a fairly simple thing to tack on here.

@burke burke force-pushed the Shopify:bundle-config branch from 76e27be to 8ae0f27 Jan 22, 2019

burke added some commits Jan 17, 2019

Insert gems into the Nix store at the correct path:
previously we would get things like:

  /nix/store/<hash>-https_rubygems_org_gems_mygem-1_2_3_gem

...while bundlerEnv would insist on reifying:

  /nix/store/<different hash>-mygem-1.2.3.gem

(the hash varies because a nix-store hash is computed over both the
contents and the name)

@burke burke force-pushed the Shopify:bundle-config branch from 8ae0f27 to 7fb2994 Jan 22, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment