Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Branch: master
Commits on Nov 17, 2011
  1. @pablohoffman

    bumped version to 0.15

    pablohoffman authored
  2. @pablohoffman

    bumped version to 0.14

    pablohoffman authored
Commits on Nov 14, 2011
  1. @pablohoffman
  2. @pablohoffman
  3. @pablohoffman
  4. @pablohoffman
Commits on Nov 8, 2011
  1. @pablohoffman
Commits on Oct 28, 2011
  1. @pablohoffman
Commits on Oct 25, 2011
  1. @pablohoffman
  2. @pablohoffman
  3. @pablohoffman
  4. @pablohoffman
Commits on Oct 23, 2011
  1. @pablohoffman
  2. @pablohoffman
  3. @pablohoffman
  4. @pablohoffman

    changed feed exports storage api so that file/stdio outputs directly …

    pablohoffman authored
    …without using a temporary file
Commits on Oct 22, 2011
  1. @pablohoffman
  2. @pablohoffman
Commits on Sep 26, 2011
  1. @pablohoffman
Commits on Sep 25, 2011
  1. @pablohoffman
Commits on Sep 23, 2011
  1. @pablohoffman
  2. @pablohoffman

    Some changes to persistent scheduler after some initial usage feedback:

    pablohoffman authored
    * added LIFO queues, in addition to the original FIFO queues
    * use LIFO queues (instead of FIFO queues) by default, since they resemble DFO
      better which is a more convenient crawling order for most cases
    * do not adjust the priority based on depth by default (DEPTH_PRIORITY = 0)
    
    If someone does need to use strict BFO order, it can be by done by setting:
    
        DEPTH_PRIORITY = 1
        SCHEDULER_DISK_QUEUE = 'scrapy.squeue.PickleFifoDiskQueue'
        SCHEDULER_MEMORY_QUEUE = 'scrapy.squeue.FifoMemoryQueue'
  3. @pablohoffman

    make SpiderState extension always available, regardless of whether th…

    pablohoffman authored
    …ere is a job dir, and make sure it always set spider.state attribute, for consistency between in-memory and on-disk runs
Commits on Sep 22, 2011
  1. @pablohoffman

    Merge pull request #42 from noplay/deploy-git-version

    pablohoffman authored
    scrapy deploy support git version
Commits on Sep 21, 2011
  1. @noplay
  2. @pablohoffman

    Merge pull request #39 from kalessin/master

    pablohoffman authored
    autothrottle fix
  3. @kalessin
Commits on Sep 18, 2011
  1. @pablohoffman
  2. @dangra

    Merge branch '0.12'

    dangra authored
  3. @dangra

    migrate hgignore to gitignore

    dangra authored
Commits on Sep 15, 2011
  1. @pablohoffman
Commits on Sep 14, 2011
  1. @pablohoffman

    Merge pull request #38 from kalessin/master

    pablohoffman authored
    _extract_links require extra parameter base_url
  2. @kalessin

    _extract_links requires extra parameter base_url in order to avoid

    kalessin authored
    exception when called from superclass method
  3. @pablohoffman

    removed documentation header notifying about other documentation vers…

    pablohoffman authored
    …ions, as that's provided by readthedocs already
  4. @pablohoffman

    removed documentation header notifying about other documentation vers…

    pablohoffman authored
    …ions, as that's provided by readthedocs already
Something went wrong with that request. Please try again.