Remove deprecated SgmlLinkExtractor and RegexLinkExtractor

[ashellunts]

BaseSgmlLinkExtractor is also removed.

Issue #4356

[codecov]

Codecov Report

Merging #4679 into master will increase coverage by 1.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4679      +/-   ##
==========================================
+ Coverage   85.16%   86.20%   +1.03%     
==========================================
  Files         162      160       -2     
  Lines        9748     9632     -116     
  Branches     1437     1414      -23     
==========================================
+ Hits         8302     8303       +1     
+ Misses       1186     1070     -116     
+ Partials      260      259       -1     

Impacted Files	Coverage Δ
scrapy/utils/trackref.py	85.71% <0.00%> (+2.85%)	⬆️

[Gallaecio]

https://github.com/scrapy/scrapy/blob/master/scrapy/linkextractors/regex.py#L7

[ashellunts]

@Gallaecio could you please advise how to proceed? Is RegexLinkExtractor still needed?
If yes, why is it subclass of class SgmlLinkExtractor? What methods from SgmlLinkExtractor should be available in RegexLinkExtractor?

[wRAR]

Yeah, this is strange, as __init__ is not overridden you will have a deprecation warning about SgmlLinkExtractor when instantiating RegexLinkExtractor.

OTOH, I just tried this and found scrapy.linkextractors.sgml imports sgmllib which is py2-only so this code just doesn't work anymore.

[Gallaecio]

In that case, maybe we should remove regex.py as well? No one must be using it in Python 3 if this is the first time we hear about it.

[wRAR]

Yeah, I almost suggested that too. We are usually very careful when removing things, but this class should have shown a deprecation warning anyway.

[ashellunts]

Ok, thank you for answers. Will update PR.

[ashellunts]

Shall I relpace RegexLinkExtractor to just LinkExtractor in these docs?

./sep/sep-016.rst:       legs = [RegexLinkExtractor(), ParseRules(), CanonicalizeUrl(), ItemIdSetter()]
./sep/sep-018.rst:       middlewares = [RegexLinkExtractor(), CallbackRules(), CanonicalizeUrl(), 

[wRAR]

/sep/ is historical so it shouldn't be changed.

I see there are conflicts, can you please resolve them?

[ashellunts]

/sep/ is historical so it shouldn't be changed.

I see there are conflicts, can you please resolve them?

Done.

[ashellunts]

Build in travis is green. But github still shows it as "in progress". 🤔

[ashellunts]

Ok, link goes to last successful build in Travis. But build for latest commit is not yet started.

[ashellunts]

Looks like travis doesn't recognize force push.

[dangra]

Now it is 🍏 (I've restarted one build to trigger the update on this PR)

[ashellunts]

Thank you @dangra.

[ashellunts]

🎉🎉🎉

[Gallaecio]

Thank you!