Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use credentials from bundler config #40

Merged
merged 6 commits into from Mar 14, 2019
Merged

Use credentials from bundler config #40

merged 6 commits into from Mar 14, 2019

Conversation

@burke
Copy link

@burke burke commented Jan 16, 2019

login:password can be specified in bundler configs.

This behaviour feels a little more in line with how bundler itself operates.

EDIT: Oh, well, this works for bundix -l but I still get a 401 in nix-build, which is puzzling, since it should have downloaded the file already.

login:password can be specified in bundler configs:

https://bundler.io/v1.15/man/bundle-config.1.html#CREDENTIALS-FOR-GEM-SOURCES

If these are specified, and the URL did not specify explicit auth,
use the authentication information from the URL.
lib/bundix/source.rb Outdated Show resolved Hide resolved
Copy link
Member

@alyssais alyssais left a comment

I'd prefer that we require "bundler" and use it to pull the data we need to fetch. Reimplementing this ourselves is brittle and won't capture the intricacies of how Bundler works. For example, Bundler allows the config file location to be customised, but with this patch authentication would fail in that case.

@burke
Copy link
Author

@burke burke commented Jan 17, 2019

Ok great, that's probably a simpler implementation anyway. I'll revise this today. 👍

@burke
Copy link
Author

@burke burke commented Jan 17, 2019

I've revised this PR to use Bundler::Settings directly, and to gracefully handle the case where bundler isn't available, for whatever reason.

With an incorrect token, then a correct one:

screen shot 2019-01-17 at 2 31 22 pm

@burke
Copy link
Author

@burke burke commented Jan 17, 2019

Though, I'm confused by this:

building '/nix/store/v408l83vns1548cwnvqa0m11xfr65zl1-gemfile-and-lockfile.drv'...
building '/nix/store/a22rs6cn6pyiql2rciygfmhrzs71vm8g-shopify-cops-0.0.11.gem.drv'...

trying https://packages.shopify.io/shopify/gems/gems/shopify-cops-0.0.11.gem
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
curl: (22) The requested URL returned error: 401 Unauthorized
error: cannot download shopify-cops-0.0.11.gem from any mirror
builder for '/nix/store/a22rs6cn6pyiql2rciygfmhrzs71vm8g-shopify-cops-0.0.11.gem.drv' failed with exit code 1
cannot build derivation '/nix/store/rdzhzhmd716pmk55d67ncf4rcrn7m6wv-ruby2.5.3-shopify-cops-0.0.11.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/2dk6sidr07wam9v99wl83x6i6y8vmq01-dev.drv': 1 dependencies couldn't be built
error: build of '/nix/store/2dk6sidr07wam9v99wl83x6i6y8vmq01-dev.drv' failed

Isn't the purpose of this code that I've modified to prefetch the *.gem files so that nix won't have to do it later? Am I misunderstanding? Is there other code to teach about bundler credentials, or is it somehow mis-predicting the derivation hash?

@alyssais
Copy link
Member

@alyssais alyssais commented Jan 19, 2019

Have you tried deleting gemset.nix? There’s something weird that happens when it already exists that I haven’t had the time to track down yet.

@burke
Copy link
Author

@burke burke commented Jan 22, 2019

The issue was that Bundix inserts into the nix-store entries like:
q3vq532pnpvq19sca3ccgi5yabi71y2x-https_rubygems_org_gems_rubocop-0_60_0_gem, while bundlerEnv seems to expect 06sp9w0i042db7y8h1gx7vwkyiw4cmpx-rubocop-0.60.0.gem.

This would cause the gems to be downloaded twice: once by bundix and then again by bundlerEnv. Since I haven't provided bundlerEnv with a way to retrieve credentials, that failed.

I've amended this PR to write files into the store at the correct paths as expected by bundlerEnv.

I also removed some dead code: I'm happy to move this to its own PR if it's easier to review that way but it seemed like a fairly simple thing to tack on here.

burke added 3 commits Jan 17, 2019
previously we would get things like:

  /nix/store/<hash>-https_rubygems_org_gems_mygem-1_2_3_gem

...while bundlerEnv would insist on reifying:

  /nix/store/<different hash>-mygem-1.2.3.gem

(the hash varies because a nix-store hash is computed over both the
contents and the name)
@burke
Copy link
Author

@burke burke commented Mar 12, 2019

(Bump) I got pulled onto another sub-project for a while but this change still seems to fix bundix for our use-case. Let me know if there's anything I can do to help shepherd this along.

@zimbatm
Copy link
Member

@zimbatm zimbatm commented Mar 13, 2019

it looks good to me on principle. It would be nice to have some tests but the infrastructure isn't there.

@burke
Copy link
Author

@burke burke commented Mar 13, 2019

Fair, I'll try to find a way to test it later this week 👍

@burke
Copy link
Author

@burke burke commented Mar 13, 2019

I've added a test for the credential injection, happy to keep revising if necessary.

@zimbatm zimbatm merged commit 21fae0e into nix-community:master Mar 14, 2019
@zimbatm
Copy link
Member

@zimbatm zimbatm commented Mar 14, 2019

brilliant, thanks @burke !

@burke burke deleted the Shopify:bundle-config branch Mar 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants