Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Progress Bar During Downloads #2

Closed
wants to merge 6 commits into from

4 participants

@ryanmelt

Hello all,
This pull request brings in a progress bar that will be displayed while downloading gems. This is particularly useful when downloading large gems such as qtbindings on Windows. If the extra STDOUT output is undesirable, it would be easy to modify it to only output for large files (maybe > 1 Mb?). I like the feedback for all file sizes personally. Let me know what you think.
Thanks,
Ryan Melton

@luislavena
Collaborator

I like it, however I'm not sure of including ProgressBar globally.

Saying this because there is a progressbar gem: https://rubygems.org/gems/progressbar and it might clash.

Rubinius guys did a simple implementation:

http://github.com/evanphx/rubinius/blob/master/configure#L241-271

That did not depend on the the additional class.

We also need to check how this could affect the automated output, specially for projects like Bundler and such.

Guys?

@ryanmelt

I added the progressbar class to the gem namespace, so it shouldn't conflict with anything now.

@luislavena
Collaborator

Hello,

I think the good idea will be remove ProgressBar at all.

Also, all the remote operations are been handled by this, so fetching of specs and everything is streamed.

There is also another problem, it doesn't work with 1.8.7:


Downloading latest_specs.4.8.gz
ERROR:  While executing gem ... (NoMethodError)oooooooooooooooo| ETA:  00:00:00
    undefined method `body=' for #Net::HTTPOK 200 OK readbody=true

Tested with ruby -Ilib bin/gem fetch qtbindings

See my patch to only generate the progress report when a gem is being downloaded: http://gist.github.com/600349

@ryanmelt

I updated it to remove the progressbar class and to use a single line that gets redrawn as suggested in your gist. Also updated to work with 1.8.7.
Ryan

@luislavena
Collaborator

Thank you Ryan

Posted about this to rubygems-devel:

http://rubyforge.org/pipermail/rubygems-developers/2010-September/005573.html

Waiting for feedback there before decide to integrate it.

@ryanmelt

Hi Luis, I believe I addressed your issue of using print and flush directly by wrapping them in say-like statements. As no one has commented, can we get this incorporated?
Thanks,
Ryan

@luislavena
Collaborator

Hello Ryan.

I bring this topic to RubyGems about making the progress bar something part of UI component to have better control of it.

Also, this print/flush approach do not work nice with remote automations like Chef, so move of this to the UI and have an option to turn it of will be the best.

See my thread and further comments there:

http://rubyforge.org/pipermail/rubygems-developers/2010-September/005573.html

This should be integrated, but to my bad, I'm not 100 percent sure on the approach and to my bad haven't received the feedback from other RubyGems maintainers

Will keep you posted.

@luislavena
Collaborator

Hello Ryan,

I'm working today on extend UserInteraction to support a configurable Download reporter, in that way, it will make more easy for non-tty interfaces to work and not generate a bunch of output.

Thank you again for your contribution. Will post a link to the commits once I'm done for you to test and review.

@qrush
Owner

Been wanting this forever. Any kind of feedback when gem install happens would be AWESOME.

@zenspider
Owner

I defer to Luis (who needs to stop deferring to the list).

@luislavena
Collaborator

Hello guys,

I've posted this to the list for review:

http://rubyforge.org/pipermail/rubygems-developers/2010-December/005942.html

I've implemented Ryan idea into a more modular and non-tty friendly version under download-reporter branch:

https://github.com/rubygems/rubygems/tree/download-reporter

Let me know your thoughts guys.

@luislavena
Collaborator

Hello Guys,

Wanted to let you know this has been merged into master (slightly different code, same idea) into master:

db2ce03

Closing this out.

Ryan, thank you for your contribution, patience and initial code for this implementation.

@jfeng5150 jfeng5150 referenced this pull request
Closed

gem install rails issue #1055

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
1  lib/rubygems.rb
@@ -92,6 +92,7 @@
# * Daniel Berger -- djberg96(at)gmail.com
# * Phil Hagelberg -- technomancy(at)gmail.com
# * Ryan Davis -- ryand-ruby(at)zenspider.com
+# * Ryan Melton -- ryanmelt(at)gmail.com
#
# (If your name is missing, PLEASE let us know!)
#
View
58 lib/rubygems/remote_fetcher.rb
@@ -304,6 +304,27 @@ def open_uri_or_path(uri, last_modified = nil, head = false, depth = 0)
end
##
+ # Returns the width of the terminal or defaults to 80 characters
+ # Modified from the Hirb gem by Gabriel Horner which is MIT licensed
+
+ def console_width
+ default_width = 80
+ begin
+ if (ENV['COLUMNS'] =~ /^\d+$/)
+ ENV['COLUMNS'].to_i
+ elsif (RUBY_PLATFORM =~ /java/ || (!STDIN.tty? && ENV['TERM'])) && command_exists?('tput')
+ `tput cols`.to_i
+ elsif STDIN.tty? && command_exists?('stty')
+ `stty size`.scan(/\d+/).map { |s| s.to_i }[1]
+ else
+ default_width
+ end
+ rescue
+ default_width
+ end
+ end
+
+ ##
# Performs a Net::HTTP request of type +request_class+ on +uri+ returning
# a Net::HTTP response object. request maintains a table of persistent
# connections to reduce connect overhead.
@@ -341,7 +362,42 @@ def request(uri, request_class, last_modified = nil)
say "#{request.method} #{uri}" if
Gem.configuration.really_verbose
- response = connection.request request
+
+ console_width_minus_1 = console_width() - 1
+ file_name = File.basename(uri.path)
+ if request.response_body_permitted? and file_name =~ /\.gem$/
+ response = connection.request request do |incomplete_response|
+ if Net::HTTPOK === incomplete_response
+ file_size = incomplete_response.content_length
+ file_size_kb = file_size / 1024
+ downloaded_size = 0
+ data = ''
+ previous_percentage = nil
+ incomplete_response.read_body do |segment|
+ data << segment
+ downloaded_size += segment.length
+ downloaded_size_kb = downloaded_size / 1024
+ downloaded_percentage = (downloaded_size * 100) / file_size
+ if downloaded_percentage != previous_percentage
+ progress_string = "\r%s [%dkB/%dkB %3d%%] " % [file_name, downloaded_size_kb, file_size_kb, downloaded_percentage]
+ progress_string << (" " * (console_width_minus_1 - progress_string.length)) if progress_string.length < console_width_minus_1
+ say_no_newline progress_string
+ say_flush()
+ end
+ previous_percentage = downloaded_percentage
+ end
+ say_no_newline "\r#{' ' * (file_name.length + 7)}\r"
+ if incomplete_response.respond_to? :body=
+ incomplete_response.body = data
+ else
+ incomplete_response.instance_variable_set(:@body, data)
+ end
+ end
+ end
+ else
+ response = connection.request request
+ end
+
say "#{response.code} #{response.message}" if
Gem.configuration.really_verbose
View
16 lib/rubygems/user_interaction.rb
@@ -116,6 +116,8 @@ module Gem::UserInteraction
:ask_yes_no,
:choose_from_list,
:say,
+ :say_no_newline,
+ :say_flush,
:terminate_interaction ].each do |methname|
class_eval %{
def #{methname}(*args)
@@ -270,6 +272,20 @@ def say(statement="")
end
##
+ # Display a statement without a carriage return.
+
+ def say_no_newline(statement="")
+ @outs.print statement
+ end
+
+ ##
+ # Flush the stream
+
+ def say_flush
+ @outs.flush
+ end
+
+ ##
# Display an informational alert. Will ask +question+ if it is not nil.
def alert(statement, question=nil)
Something went wrong with that request. Please try again.