Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Add support for progress formatters #8

Closed
wants to merge 4 commits into from

2 participants

@RyanScottLewis

Rspec style formatters for the output of gem dependant

@grosser grosser commented on the diff
lib/rubygems/dependent.rb
@@ -29,11 +39,16 @@ def self.find(gem, options={})
private
def self.fetch_all_dependencies(specs_and_sources, options={})
- parallel = (options[:parallel] || 15)
+ parallel = (options[:parallel] || 15) # NOTE: 15? A general rule is about 1-4 threads per CPU core
@grosser Owner
grosser added a note

Depends on how much blocking is going on, this is not high performance computing :)
Maybe add a little benchmark script if you want another number.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@grosser
Owner

Just throw out the old formatter, progress is nicer :)
Can you do without activesupport, it's such a giant lib ...

print "1/1000 - 0%"
print "\r"
print "2/1000 - 0%"
@RyanScottLewis

I'll look around for an alternative to the ActiveSupport. I'm really only using it for the demodulize and the underscore String inflections.

I actually kinda liked the 'dot' formatter. Because of the threading, the percentage formatter gets a little screwy and will actually look like the percentage is going down for a second.

The next formatter is going to be a progress bar. When I add that, I'm also going to add "on_error" and "on_kill_thread" (maybe named differently) hooks for the ProgressFormatter class. But I gotta work out the kinks on the threading.

@grosser grosser closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
1  .gitignore
@@ -0,0 +1 @@
+.rvmrc
View
1  Gemfile
@@ -3,3 +3,4 @@ gemspec
gem 'rake'
gem 'rspec', '~>2'
+gem 'active_support'
View
4 Gemfile.lock
@@ -6,6 +6,9 @@ PATH
GEM
remote: http://rubygems.org/
specs:
+ active_support (3.0.0)
+ activesupport (= 3.0.0)
+ activesupport (3.0.0)
diff-lcs (1.1.2)
rake (0.8.7)
rspec (2.0.1)
@@ -23,6 +26,7 @@ PLATFORMS
ruby
DEPENDENCIES
+ active_support
gem-dependent!
rake
rspec (~> 2)
View
10 lib/rubygems/commands/dependent_command.rb
@@ -4,7 +4,7 @@ class Gem::Commands::DependentCommand < Gem::Command
def initialize
super 'dependent', 'Show which gems depend on a gem', :progress => true
- add_option('--source URL', 'Query these sources (e.g. http://gemcutter.org)') do |n, _|
+ add_option('-s', '--source URL', 'Query these sources (e.g. http://rubygems.org)') do |n, _|
options[:source] = n.to_s.split(',')
end
@@ -12,11 +12,15 @@ def initialize
options[:progress] = false
end
- add_option('--fetch-limit N', Integer, 'Fetch specs for max N gems (for fast debugging)') do |n, _|
+ add_option('-f', '--progress-format [FORMAT]', String, 'Format of progress output (dot, percentage)') do |f|
+ options[:progress_format] = f.downcase.strip.to_sym
+ end
+
+ add_option('-l', '--fetch-limit N', Integer, 'Fetch specs for max N gems (for fast debugging)') do |n, _|
options[:fetch_limit] = n
end
- add_option('--parallel N', Integer, 'Make N requests in parallel') do |n, _|
+ add_option('-p', '--parallel N', Integer, 'Make N requests in parallel') do |n, _|
options[:parallel] = n
end
View
39 lib/rubygems/dependent.rb
@@ -3,25 +3,35 @@
require 'rubygems/spec_fetcher'
module Gem
- class Dependent
+ module Dependent
+
def self.find(gem, options={})
+ options[:progress] ||= true
+
+ if options[:progress]
+ options[:progress_format] ||= :dot
+
+ require 'rubygems/dependent/progress_formatter' # Lazily load formatters
+
+ formatter = ProgressFormatter.find(options[:progress_format]).new
+ end
+
# get all gems
specs_and_sources = with_changed_gem_source(options[:source]) do
all_specs_and_sources(:all_versions => options[:all_versions])
end
- if options[:fetch_limit]
- specs_and_sources = specs_and_sources.first(options[:fetch_limit])
- end
+ specs_and_sources = specs_and_sources.first(options[:fetch_limit]) if options[:fetch_limit]
- if options[:progress]
- $stderr.puts "Downloading specs for #{specs_and_sources.size} gems"
- end
+ session = { :options => options, :count => specs_and_sources.size }
- gems_and_dependencies = fetch_all_dependencies(specs_and_sources, options) do
- print_dot if options[:progress]
+ formatter.on_init(session) if options[:progress] && formatter.respond_to?(:on_init)
+
+ gems_and_dependencies = fetch_all_dependencies(specs_and_sources, options) do |gem|
+ formatter.on_download(gem) if options[:progress] && formatter.respond_to?(:on_download)
end
- $stderr.print "\n" if options[:progress]
+
+ formatter.on_complete if options[:progress] && formatter.respond_to?(:on_complete)
select_dependent(gems_and_dependencies, gem)
end
@@ -29,11 +39,16 @@ def self.find(gem, options={})
private
def self.fetch_all_dependencies(specs_and_sources, options={})
- parallel = (options[:parallel] || 15)
+ parallel = (options[:parallel] || 15) # NOTE: 15? A general rule is about 1-4 threads per CPU core
@grosser Owner
grosser added a note

Depends on how much blocking is going on, this is not high performance computing :)
Maybe add a little benchmark script if you want another number.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Gem::Dependent::Parallel.map(specs_and_sources, :in_processes => parallel) do |spec, source|
- yield if block_given?
name, version = spec[0,2]
+
+ gem = { :spec => spec, :source => source, :name => name, :version => version }
+
+ yield(gem) if block_given?
+
dependencies = fetch_dependencies(spec, source, options)
+
[name, version, dependencies]
end
end
View
2  lib/rubygems/dependent/parallel.rb
@@ -7,7 +7,7 @@
require 'rbconfig'
module Gem;end
-class Gem::Dependent;end
+module Gem::Dependent;end
class Gem::Dependent::Parallel
VERSION = '0.5.8'
View
62 lib/rubygems/dependent/progress_formatter.rb
@@ -0,0 +1,62 @@
+require 'active_support/core_ext/string/inflections'
+
+module Gem::Dependent
+ # A namespace to define all progress formatters
+ module ProgressFormatters; end
+
+ class ProgressFormatter
+ # NOTE: Can't decide on `on_init`, `on_begin`, `on_start`, etc.. `on_init` seems to be the most popular for event-style classes.
+
+ class NotFound < StandardError
+ def to_s
+ available_formatters = ProgressFormatter.all.keys.sort
+
+ "\nFormatter not found\nAvailable formatters: #{available_formatters.join(', ')}\n\n"
+ end
+ end
+
+ class << self
+
+ # An Array containing all immediate children of Gem::Dependent::Formatter
+ attr :all
+
+ # Add any immediate children of Gem::Dependent::Formatter to an Hash that
+ # we can easily access by using Gem::Dependent::Formatter.all
+ #
+ # The key is the name of the child as a formatted Symbol and the value is
+ # the child
+ def inherited(child)
+ (@all ||= {})[ child.to_s.demodulize.underscore.to_sym ] = child
+ end
+
+ # Find a child Class of Gem::Dependent::ProgressFormatter by it's formatted Symbol
+ def find(child_name)
+ result = @all[child_name]
+
+ raise NotFound unless result
+
+ result
+ end
+
+ end
+
+ def print(*args)
+ $stderr.print(*args)
+ # flush
+ end
+
+ def puts(*args)
+ $stderr.puts(*args)
+ # flush
+ end
+
+ # Make progress visible
+ def flush
+ $stderr.flush if rand(20) == 0
+ end
+ end
+
+end
+
+# Load all formatters
+Dir[ File.join( File.expand_path(File.dirname(__FILE__)), 'progress_formatters', '*') ].each { |filename| require filename }
View
17 lib/rubygems/dependent/progress_formatters/dot.rb
@@ -0,0 +1,17 @@
+module Gem::Dependent::ProgressFormatters
+
+ class Dot < Gem::Dependent::ProgressFormatter
+ def on_init(session)
+ puts "Downloading specs for #{session[:count]} gems"
+ end
+
+ def on_download(gem)
+ print "."
+ end
+
+ def on_complete
+ puts
+ end
+ end
+
+end
View
22 lib/rubygems/dependent/progress_formatters/percentage.rb
@@ -0,0 +1,22 @@
+module Gem::Dependent::ProgressFormatters
+
+ class Percentage < Gem::Dependent::ProgressFormatter
+ def percent_complete
+ '%.2f' % ((@complete.to_f/@count.to_f)*100.0)
+ end
+
+ def on_init(session)
+ @count = session[:count]
+ @complete = 0
+
+ puts "Downloading specs for #{@count} gems"
+ end
+
+ def on_download(gem)
+ @complete += 1
+
+ print "#{percent_complete}% \r"
+ end
+ end
+
+end
View
2  lib/rubygems/dependent/version.rb
@@ -1,5 +1,5 @@
module Gem
- class Dependent
+ module Dependent
VERSION = Version = '0.2.2'
end
end
View
0  spec/fixtures/gemcutter_specs.yml → spec/fixtures/rubygems_specs.yml
File renamed without changes
Something went wrong with that request. Please try again.