Permalink
Browse files

CLI and HTTP proxy

Adds a command-line tool for scanning local files as well as an HTTP proxy.

The HTTP proxy can be used by starting it up:

    $ deadweight -l dw.log -s styles.css -w http://mojodna.net/ -P

And configuring Firefox (etc.) to use it ("Manual Proxy Configuration") before
browsing through your site. It will helpfully display unmatched rules in your
terminal window so you can see what to look for. `-w` configures a whitelist so
you don't match your stylesheets against unexpected sites.

Under OS X, you may be able to use [Lyndon](http://github.com/defunkt/lyndon)
(with the `-L` option) to parse pages that contain HTML-modifying Javascript.
  • Loading branch information...
1 parent f909bbe commit 589bc4f7bd2d294a40a2702fc08b21bb4235ad78 @mojodna mojodna committed with Aug 31, 2009
Showing with 218 additions and 3 deletions.
  1. +14 −1 README.rdoc
  2. +5 −0 bin/deadweight
  3. +4 −2 deadweight.gemspec
  4. +175 −0 lib/deadweight/cli.rb
  5. +20 −0 test/cli_test.rb
View
@@ -20,7 +20,19 @@ Ryan Bates has worked his magic once again. Head over here for an excellent intr
puts dw.run
end
-This will output all unused selectors, one per line.
+This will output all unused rules, one per line.
+
+Alternately, you can run it from the command-line:
+
+ $ deadweight -s styles.css -s ie.css index.html about.html
+
+You can pipe in CSS rules from STDIN:
+
+ $ cat styles.css | deadweight index.html
+
+And you can use it as an HTTP proxy:
+
+ $ deadweight -l deadweight.log -s styles.css -w http://github.com/ -P
=== How You Install It
@@ -32,6 +44,7 @@ This will output all unused selectors, one per line.
- By default, it looks at http://localhost:3000.
- It's completely dumb about any classes, IDs or tags that are only added by your Javascript layer, but you can filter them out by setting +ignore_selectors+.
- You can optionally tell it to use Mechanize, and set up more complicated targets for scraping by specifying them as Procs.
+- There is experimental support for Lyndon (http://github.com/defunkt/lyndon) with -L
=== A More Complex Example, In Light of All That
View
@@ -0,0 +1,5 @@
+#!/usr/bin/env ruby
+require 'deadweight/cli'
+
+Deadweight::CLI.execute(STDOUT, STDIN, STDERR, ARGV.dup)
+
View
@@ -7,20 +7,22 @@ Gem::Specification.new do |s|
s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
s.authors = ["Aanand Prasad"]
s.date = %q{2009-08-18}
+ s.default_executable = %q{deadweight}
s.email = %q{aanand.prasad@gmail.com}
+ s.executables = %q{deadweight}
s.extra_rdoc_files = [
"LICENSE",
"README.rdoc"
]
s.files = [
- ".document",
- ".gitignore",
"LICENSE",
"README.rdoc",
"Rakefile",
"VERSION",
"deadweight.gemspec",
+ "bin/deadweight",
"lib/deadweight.rb",
+ "lib/deadweight/cli.rb",
"test/deadweight_test.rb",
"test/fixtures/index.html",
"test/fixtures/index2.html",
View
@@ -0,0 +1,175 @@
+require 'optparse'
+require 'deadweight'
+
+class Deadweight
+ class CLI
+ attr_reader :stdout, :stdin, :stderr
+ attr_reader :arguments, :options
+ attr_reader :output
+
+ def self.execute(stdout, stdin, stderr, arguments = [])
+ @options = {
+ :log_file => stderr,
+ :output => stdout,
+ :proxy_port => 8002
+ }
+
+ self.parse_options(arguments)
+ self.new(stdout, stdin, stderr, arguments, @options).execute!
+ end
+
+ def self.option_parser
+ @option_parser ||= OptionParser.new do |opts|
+ opts.banner = "Usage: #{$0} [options] <url> [<url> ...]"
+
+ @options[:stylesheets] = []
+
+ opts.on("-L", "--lyndon", "Pre-process HTML with Lyndon") do
+ @options[:lyndon] = true
+ end
+
+ opts.on("-l", "--log FILE", "Where to write log messages") do |v|
+ @options[:log_file] = v
+ end
+
+ opts.on("-P", "--proxy", "Run in proxy mode") do
+ @options[:proxy] = true
+ end
+
+ opts.on("-p", "--proxy-port", "Port to run the proxy on") do |v|
+ @options[:proxy] = true
+ @options[:proxy_port] = v.to_i
+ end
+
+ opts.on("-O", "--output FILE",
+ "Where to output orphaned CSS rules") do |v|
+ @options[:output] = File.new(v, "w")
+ end
+
+ opts.on("-s", "--stylesheet FILE",
+ "Apply the specified stylesheet to the target") do |v|
+ @options[:stylesheets] << v
+ end
+
+ opts.on("-w", "--whitelist URL-PREFIX",
+ "Specifies a prefix for URLs to process") do |v|
+ @options[:whitelist] ||= []
+ @options[:whitelist] << v
+ end
+ end
+ end
+
+ def self.parse_options(arguments = [])
+ self.option_parser.parse!(arguments)
+ @options
+ end
+
+ def initialize(stdout, stdin, stderr, arguments = [], options = {})
+ @stdout = stdout
+ @stdin = stdin
+ @stderr = stderr
+ @arguments = arguments
+ @options = options
+ @output = options[:output]
+ end
+
+ def execute!
+ if options[:proxy]
+ proxy
+ elsif arguments.empty?
+ stdout.puts self.class.option_parser.help
+ else
+ process
+ end
+ end
+
+ def process
+ # TODO pass stylesheets + pages as args
+ dw = Deadweight.new
+
+ # TODO this should be the default
+ dw.root = ""
+
+ dw.log_file = options[:log_file]
+
+ dw.stylesheets = options[:stylesheets]
+
+ dw.rules = stdin.read if stdin.stat.size > 0
+
+ if options[:lyndon]
+ arguments.each do |file|
+ dw.pages << IO.popen("cat #{file} | lyndon 2> /dev/null")
+ end
+ else
+ dw.pages = arguments
+ end
+
+ unused_rules = dw.run
+ unused_rules.each do |k,v|
+ output.puts "#{k} { #{v} }"
+ end
+ end
+
+ def proxy
+ dw = Deadweight.new
+
+ # TODO note the boilerplate shared with #process
+ dw.root = ""
+ dw.log_file = options[:log_file]
+ dw.stylesheets = options[:stylesheets]
+ dw.rules = stdin.read if stdin.stat.size > 0
+
+ # initialize selectors
+ dw.run
+
+ require 'webrick/httpproxy'
+
+ @proxy = WEBrick::HTTPProxyServer.new \
+ :AccessLog => [
+ [options[:log_file], WEBrick::AccessLog::COMMON_LOG_FORMAT],
+ [options[:log_file], WEBrick::AccessLog::REFERER_LOG_FORMAT]
+ ],
+ :Logger => WEBrick::Log.new(options[:log_file]),
+ :Port => options[:proxy_port],
+ :ProxyContentHandler => lambda { |request, response|
+
+ parse_this = false
+
+ if options[:whitelist]
+ p options[:whitelist]
+ options[:whitelist].each do |x|
+ if response.request_uri.to_s[0..x.length - 1].downcase == x.downcase
+ parse_this = true
+ break
+ end
+ end
+ else
+ parse_this = true
+ end
+
+ if parse_this
+ dw.process!(response.body)
+
+ stdout.puts "After reviewing <#{response.request_uri}>, these were left:"
+ dw.unused_selectors.each do |k,v|
+ stdout.puts "#{k} { #{v} }"
+ end
+ end
+ }
+
+ trap('INT') do
+ @proxy.shutdown
+
+ # dump the remaining CSS rules if output is set
+ unless options[:output] == STDOUT
+ dw.unused_selectors.each do |k,v|
+ output.puts "#{k} { #{v} }"
+ end
+ end
+ end
+
+ @proxy.start
+ end
+ end
+end
+
View
@@ -0,0 +1,20 @@
+require 'test_helper'
+
+class CliTest < Test::Unit::TestCase
+ COMMAND = "ruby -rubygems -Ilib bin/deadweight -s test/fixtures/style.css test/fixtures/index.html 2>/dev/null"
+
+ should "output unused selectors on STDOUT" do
+ @result = `#{COMMAND}`.split("\n")
+
+ assert_equal 1, @result.grep(/^#foo \.bar \.baz \{/).length
+ assert_equal 0, @result.grep(/^#foo \{/).length
+ assert_equal 0, @result.grep(/^#foo .bar \{/).length
+ end
+
+ should "accept CSS rules on STDIN" do
+ @result = `echo ".something { display: block; }" | #{COMMAND}`.split("\n")
+
+ assert_equal 1, @result.grep(/^\.something \{/).length
+ end
+end
+

0 comments on commit 589bc4f

Please sign in to comment.