Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull all the gems #48

Closed
wants to merge 23 commits into from
Closed

Pull all the gems #48

wants to merge 23 commits into from

Conversation

pcarranza
Copy link
Contributor

This fixes #18 by adding a thor client to run this process.

Check the reference documentation for details on how to use it.

method_option :latest, :type => :boolean, :default => false, :desc =>
"Only fetch the latest specs"
method_option :server_url, :type => :string, :default => "http://localhost:9292", :desc =>
"Gemstash server url"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I'm reading the code right, it looks like this is the upstream URL, not the Gemstash URL. If that's the case, that seems like it conflicts with the task description discussing the default upstream. If this is the upstream, I think it should probably be nil by default, so the task description will be correctly pulling from the default upstream when this is nil (which may require some tweaks to the code).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, it is not the upstream url, I do create an Upstream just to reuse the object with the client (https://github.com/bundler/gemstash/pull/48/files#diff-b6f92f551dd149bd7013ca6f7f0fd7a2R17)

What this does is create an HTTPClient and start requesting the head of the gems to gemstash without using any upstream path, so it downloads those from the default upstream. This process will not make any call to rubygems.org directly.

All this does is fake a client for every gem version there is in the specs file.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, gotchya

@pcarranza
Copy link
Contributor Author

Thanks for the feedback.

Next step, I'll give it a spin to use the parallel gem instead of homebrewed thread pool and change the head method for GemFetcher direct call, will see how it works.

@pcarranza
Copy link
Contributor Author

At this point I'm still using head to download the gems.

What would be left now is extracting the gem fetching and storing out of the upstream source so I can build one and not use the server. As is it is capable of fetching all the gems (all, prerelease and latest) just that it requires gemstash to be there to hit it.

@smellsblue smellsblue added this to the Version 1.1.0 milestone Feb 20, 2016
@@ -313,11 +322,86 @@ Show what version of Gemstash you are using.
### Usage

```
<<<<<<< HEAD
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Um, is this a Git conflict marker?) Just a fan reading cool code, and I noticed it.

@pcarranza pcarranza closed this Apr 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Task to actively pre-cache all of rubygems.org
3 participants