Provide complete API documentation coverage of scrapy.linkextractors

[Gallaecio]

I made the assumption that FilteringLinkExtractor was originally created for code sharing between the implementations based on lxml and sgmllib.

Here I’m marking FilteringLinkExtractor as deprecated, assuming that upon removal its code can go into LxmlLinkExtractor.

I’m also updating the documentation about link extractors to talk about LxmlLinkExtractor only, without even mentioning the possibility of alternative implementations.

[codecov]

Codecov Report

Merging #4045 into master will decrease coverage by 0.08%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #4045      +/-   ##
==========================================
- Coverage   83.85%   83.77%   -0.09%     
==========================================
  Files         165      165              
  Lines        9615     9634      +19     
  Branches     1440     1442       +2     
==========================================
+ Hits         8063     8071       +8     
- Misses       1304     1311       +7     
- Partials      248      252       +4

Impacted Files	Coverage Δ
scrapy/linkextractors/lxmlhtml.py	92.3% <ø> (ø)	⬆️
scrapy/linkextractors/__init__.py	97.01% <100%> (+0.34%)	⬆️
scrapy/robotstxt.py	75.3% <0%> (-22.23%)	⬇️
scrapy/core/downloader/__init__.py	89.31% <0%> (-1.53%)	⬇️
scrapy/core/downloader/handlers/http11.py	92.99% <0%> (+0.08%)	⬆️
scrapy/utils/trackref.py	85.71% <0%> (+2.85%)	⬆️
scrapy/spiders/crawl.py	92.47% <0%> (+10.33%)	⬆️

[elacuesta]

Looks good to me, except for the following:

I’m also updating the documentation about link extractors to talk about LxmlLinkExtractor only, without even mentioning the possibility of alternative implementations.

Personally I don't remember seeing a custom Link extractor implementation, but I'm not sure about removing that section. It doesn't hurt, it doesn't need any particular code to be handled and it's technically still supported.
Just a comment, as I said I haven't seen or written any custom extractor so my opinion on the subject is not that strong.

[Gallaecio]

If we leave it, I would like to at least mention a few use cases, and show a code example. But I cannot think of any use case myself.

[Gallaecio]

@kmike What do you think, should custom link extractors remain in the documentation or go?

[kmike]

I like the simplification of the docs, both the wording and content. +1 to deprecate / un-expose FilteringLinkExtractor. As I recall, it was needed when we had several link extractors, but now this code is all way to complex for what it is doing.

[kmike]

Looks good, thanks @Gallaecio! I left a small comment about a class name used in tests, but otherwise it looks ready to merge 👍

[kmike]

Thanks @Gallaecio!