Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documentation of link extractor usage #4775

Merged
merged 9 commits into from Oct 1, 2020

Conversation

HAKSOAT
Copy link
Contributor

@HAKSOAT HAKSOAT commented Sep 2, 2020

Added description when using link extractor outside crawlspider and created reference documentation for scrapy.link.Link class

Fixes #4751

…created reference documentation for scrapy.link.Link class
@codecov
Copy link

codecov bot commented Sep 3, 2020

Codecov Report

Merging #4775 into master will increase coverage by 0.19%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4775      +/-   ##
==========================================
+ Coverage   87.01%   87.21%   +0.19%     
==========================================
  Files         160      160              
  Lines        9812     9813       +1     
  Branches     1447     1447              
==========================================
+ Hits         8538     8558      +20     
+ Misses       1013      994      -19     
  Partials      261      261              
Impacted Files Coverage Δ
scrapy/link.py 100.00% <ø> (ø)
scrapy/cmdline.py 67.20% <0.00%> (-0.55%) ⬇️
scrapy/pqueues.py 99.08% <0.00%> (ø)
scrapy/utils/conf.py 93.69% <0.00%> (ø)
scrapy/spiderloader.py 100.00% <0.00%> (ø)
scrapy/core/spidermw.py 100.00% <0.00%> (ø)
scrapy/utils/template.py 100.00% <0.00%> (ø)
scrapy/utils/trackref.py 82.85% <0.00%> (ø)
scrapy/pipelines/files.py 60.64% <0.00%> (ø)
scrapy/pipelines/media.py 97.16% <0.00%> (ø)
... and 14 more

docs/topics/link-extractors.rst Outdated Show resolved Hide resolved
docs/topics/link.rst Outdated Show resolved Hide resolved
Copy link
Member

@Gallaecio Gallaecio left a comment

Unless we decide to cover other use cases (and I cannot think of any at the moment), I would move the new documentation into the existing topic about link extractors, instead of creating a new page.

@HAKSOAT
Copy link
Contributor Author

HAKSOAT commented Sep 10, 2020

Unless we decide to cover other use cases (and I cannot think of any at the moment), I would move the new documentation into the existing topic about link extractors, instead of creating a new page.

Done. @Gallaecio Can you help take a look at what it looks like now? Perhaps a couple more changes to make it better?

scrapy/link.py Outdated Show resolved Hide resolved
docs/topics/link-extractors.rst Outdated Show resolved Hide resolved
HAKSOAT and others added 2 commits Sep 16, 2020
Co-authored-by: Adrián Chaves <adrian@chaves.io>
Improvements to URL description
scrapy/link.py Show resolved Hide resolved
HAKSOAT added 3 commits Sep 16, 2020
Fixed line length Flake issue
Fixed trailing whitespace
Copy link
Member

@Gallaecio Gallaecio left a comment

Looks good to me, thanks!

@kmike
Copy link
Member

kmike commented Oct 1, 2020

Thanks @HAKSOAT!

@kmike kmike merged commit f47b120 into scrapy:master Oct 1, 2020
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Document link extractor usage outside CrawlSpider rules
5 participants