Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[MRG+1] Fix for KeyError in robots.txt middleware #1735

Merged
merged 3 commits into from Feb 3, 2016
Merged

[MRG+1] Fix for KeyError in robots.txt middleware #1735

merged 3 commits into from Feb 3, 2016

Commits on Jan 27, 2016

  1. Add test for already failed deferreds when downloading page in robots… 

    ….txt middleware.
    @ArturGaspar
    ArturGaspar committed Jan 27, 2016
    Copy the full SHA
    dc8701e View commit details
    Browse the repository at this point in the history

  2. Fix handling of already failed deferreds when downloading page in rob… 

    …ots.txt middleware.
    @ArturGaspar
    ArturGaspar committed Jan 27, 2016
    Copy the full SHA
    b2beb3e View commit details
    Browse the repository at this point in the history

  3. Do not forget failed requests in robots.txt middleware.

    @ArturGaspar
    ArturGaspar committed Jan 27, 2016
    Copy the full SHA
    90e3ae1 View commit details
    Browse the repository at this point in the history