Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update link for reasoning value of URLLENGTH_LIMIT #5134

Merged
merged 1 commit into from
May 6, 2021
Merged

Update link for reasoning value of URLLENGTH_LIMIT #5134

merged 1 commit into from
May 6, 2021

Conversation

rennerocha
Copy link
Contributor

Previous link doesn't exist anymore.
Looking into the content of archived content (https://web.archive.org/web/20180308145804/https://boutell.com/newfaq/misc/urllength.html) I realized that this default value is related to the article of Microsoft support.
Perhaps we should reconsider this default (or this middleware in future), but at least it will be good to have an updated doc.

Previous link doesn't exist anymore.
Looking into the content of archived content (https://web.archive.org/web/20180308145804/https://boutell.com/newfaq/misc/urllength.html) I realized that this default value is related to the article of Microsoft support.
Perhaps we should reconsider this default (or this middleware in future), but at least it will be good to have an updated doc.
@codecov
Copy link

codecov bot commented May 6, 2021

Codecov Report

Merging #5134 (3a95ad5) into master (e27eff4) will decrease coverage by 0.17%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5134      +/-   ##
==========================================
- Coverage   88.27%   88.10%   -0.18%     
==========================================
  Files         162      162              
  Lines       10474    10474              
  Branches     1517     1517              
==========================================
- Hits         9246     9228      -18     
- Misses        953      971      +18     
  Partials      275      275              
Impacted Files Coverage Δ
scrapy/robotstxt.py 75.30% <0.00%> (-22.23%) ⬇️

@Gallaecio Gallaecio merged commit 34b2162 into scrapy:master May 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants