Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup settings._DictProxy and scrapy.telnet #5730

Merged
merged 3 commits into from
Nov 23, 2022
Merged

Conversation

burneyh
Copy link
Contributor

@burneyh burneyh commented Nov 23, 2022

This change removes the _DictProxy class from scrapy/settings/__init__.py since it is no longer used. It also removes scrapy.telnet from the DEPRECATION_RULES as part of the cleanup. It resolves #5729, resolves #5725.

@burneyh burneyh changed the title Remove settings._DictProxy Cleanup settings._DictProxy and scrapy.telnet Nov 23, 2022
DEPRECATION_RULES = [
('scrapy.telnet.', 'scrapy.extensions.telnet.'),
]
DEPRECATION_RULES = []


def update_classpath(path):
Copy link
Member

@Laerte Laerte Nov 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should remove this function as well, since now it will just return the provided path, make sense @wRAR?

And probably pass a lambda function to convert parameter below (and remove from tests).

def build_component_list(compdict, custom=None, convert=update_classpath):

Copy link
Contributor Author

@burneyh burneyh Nov 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there is a possibility of the DEPRECATION_RULES list being repopulated in the future due to further deprecation of packages then I would recommend to keep the update_classpath method otherwise similar logic will need to be re-written.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be not the case since @deprecated decorator already have this functionality:

def deprecated(use_instead=None):

But let's see what Scrapy team has to say about.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update_classpath is used in the code that parses object dicts such as DOWNLOADER_MIDDLEWARES:

def build_component_list(compdict, custom=None, convert=update_classpath):

I think you can't use @deprecated for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah but the logic depends all on DEPRECATION_RULES variable right?

    for prefix, replacement in DEPRECATION_RULES:
        if isinstance(path, str) and path.startswith(prefix):
            new_path = path.replace(prefix, replacement, 1)
            warnings.warn(f"`{path}` class is deprecated, use `{new_path}` instead",
                          ScrapyDeprecationWarning)
            return new_path
    return path

If nothing is provided it will just return what was originally send by parameter right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not against removing it.

However, then we would need to restore it if we deprecate a component path in the future, and chances are whoever does it will not realize there was a previous implementation that can be restored, and will waste time coming up with a new implementation, possibly worse than this one.

So keeping this code around, even if currently unused, may be the lesser evil.

@codecov
Copy link

codecov bot commented Nov 23, 2022

Codecov Report

Merging #5730 (5093020) into master (45c2bd7) will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #5730      +/-   ##
==========================================
+ Coverage   88.73%   88.78%   +0.04%     
==========================================
  Files         162      162              
  Lines       10964    10949      -15     
  Branches     1799     1799              
==========================================
- Hits         9729     9721       -8     
+ Misses        954      946       -8     
- Partials      281      282       +1     
Impacted Files Coverage Δ
scrapy/settings/__init__.py 98.19% <ø> (+4.78%) ⬆️
scrapy/utils/deprecate.py 95.45% <100.00%> (+0.06%) ⬆️
scrapy/core/downloader/__init__.py 90.97% <0.00%> (-1.51%) ⬇️

@burneyh burneyh closed this Nov 23, 2022
@burneyh burneyh reopened this Nov 23, 2022
@burneyh
Copy link
Contributor Author

burneyh commented Nov 23, 2022

@wRAR, is this PR ready to be merged?

@wRAR
Copy link
Member

wRAR commented Nov 23, 2022

@hanzallah we need two approvals from the core contributors to merge a PR.

@Gallaecio Gallaecio self-requested a review November 23, 2022 12:55
@Gallaecio Gallaecio merged commit f6e9e65 into scrapy:master Nov 23, 2022
@Gallaecio
Copy link
Member

Great, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove settings._DictProxy Remove scrapy.telnet from scrapy.utils.deprecate.DEPRECATION_RULES
4 participants