Permalink
Switch branches/tags
Nothing to show
Commits on Jun 3, 2011
  1. Get rid of the graph from git log --stat

    bronson committed Jun 3, 2011
    If a file changes size, the graph changes for EVERY file.
    Makes it very hard to read the diffs.
  2. Special case: unzip using 7z for a checklist.vim package.

    bronson committed Jun 3, 2011
    We should probably convert to always using 7z.  Debian's unzip
    command is failing to keep up with the times.
  3. Fix extension claiming tarfile was gzipped but it wasn't.

    bronson committed Jun 3, 2011
    Needed by 3576 - jshint.vim
Commits on Apr 18, 2011
  1. Revert "do not render scripts.html"

    bronson committed Apr 18, 2011
    Keep rendering scripts.html until we have a page that replaces it.
    This reverts commit 471d5f5.
  2. fix .json generation

    gmarik committed with bronson Apr 16, 2011
  3. do not render scripts.html

    gmarik committed with bronson Apr 16, 2011
    - instead /api/scripts_recent.json will be used to render that page
    dynamically
Commits on Apr 7, 2011
Commits on Mar 26, 2011
Commits on Mar 24, 2011
  1. ignore 3519, it's just garbage

    bronson committed Mar 24, 2011
Commits on Mar 20, 2011
Commits on Mar 17, 2011
  1. Add a comment about parse errors

    bronson committed Mar 17, 2011
Commits on Feb 24, 2011
  1. they're retries not tries

    bronson committed Feb 24, 2011
Commits on Feb 23, 2011
  1. Revert "get rid of webcache"

    bronson committed Feb 23, 2011
    Turns out the webcache is really useful for testing scraping.
    And it's vital to fully test scraping after updating gems.
    This reverts commit 63ca733.
    
    Conflicts:
    
    	scraper
  2. Revert hpricot and restore the monkeypatch.

    bronson committed Feb 23, 2011
    Otherwise, I get this error:
    
    /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/builder.rb:9:in `gsub': invalid byte sequence in UTF-8 (ArgumentError)
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/builder.rb:9:in `uxs'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/tag.rb:139:in `to_s'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/traverse.rb:160:in `block in inner_text'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/traverse.rb:160:in `map'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/traverse.rb:160:in `inner_text'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/traverse.rb:160:in `block in inner_text'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hpricot-0.8.3/lib/hpricot/traverse.rb:160:in `map'
  3. Revert "new hpricot means no more need for a monkeypatch, yay!"

    bronson committed Feb 23, 2011
    I didn't test enough scraping.  :(
    This reverts commit cb3b6ff.
  4. downgrade activesupport to work around some bizarre gem interaction.

    bronson committed Feb 23, 2011
    With this Gemfile:
    
      source 'http://rubygems.org'
      gem 'json', '= 1.4.6'
      gem 'hashie', '= 0.2.2'
      gem 'activesupport', '= 3.0.2'
      gem 'feedzirra', '= 0.0.24'
      gem 'i18n', '= 0.4.2'
    
    Run this script:
    
      require 'bundler'
      Bundler.require
      require 'json'
      require 'hashie'
      JSON.pretty_generate(Hashie::Mash.new)
    
    And see this error:
    
    /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.2/lib/active_support/json/encoding.rb:61:in `options_for': undefined method `merge' for #<JSON::Ext::Generator::State:0xa154e38> (NoMethodError)
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.2/lib/active_support/json/encoding.rb:46:in `block in encode'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.2/lib/active_support/json/encoding.rb:77:in `check_for_circular_references'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.2/lib/active_support/json/encoding.rb:45:in `encode'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.2/lib/active_support/json/encoding.rb:30:in `encode'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/activesupport-3.0.2/lib/active_support/core_ext/object/to_json.rb:20:in `to_json'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/hashie-0.2.2/lib/hashie/hash.rb:19:in `to_json'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/json-1.4.6/lib/json/common.rb:264:in `generate'
      from /home/bronson/.rvm/gems/ruby-1.9.2-p0/gems/json-1.4.6/lib/json/common.rb:264:in `pretty_generate'
      from console:5:in `<main>'
  5. Update gems

    bronson committed Feb 23, 2011
    Can't update MimeMagic since the new PDF magic is too loose.
Commits on Feb 22, 2011
  1. more items for the todo

    bronson committed Feb 22, 2011
Commits on Feb 21, 2011
  1. add interactive console

    gmarik committed Feb 21, 2011
    - allows running an IRB console:
      $ ./console
      irb> script = JSON.parse(File.read("scripts/3458 - vundle.json"))
      irb> version = script['versions'].first
      irb> pkgfile = download_package(version, script)
      => "packages/3458 - vundle/2011-02-19 0.5 vundle.zip"
Commits on Feb 20, 2011
Commits on Feb 17, 2011
Commits on Dec 19, 2010
  1. some comment updates

    bronson committed Dec 19, 2010
Commits on Dec 8, 2010
  1. log corrupt vimballs

    bronson committed Dec 8, 2010
  2. Use native unvimball

    bronson committed Dec 8, 2010
    From carlhuda/janus@6ae5611
    Cuts extraction time by 60% (which is silly since vimballs comprise
    barely 6% of all packages) and removes lots of integration pain points.
    Vimballs still suck though.
Commits on Nov 24, 2010
  1. restore retrying to scrape

    bronson committed Nov 24, 2010
    fails 2 or 3 times during scrape without it