Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed RecursionError caused by giving config_from_object nested mod… #8619

Merged
merged 2 commits into from
Nov 8, 2023

Conversation

frolenkov-nikita
Copy link
Contributor

@frolenkov-nikita frolenkov-nikita commented Nov 7, 2023

Fixed RecursionError caused by giving config_from_object nested module that does not exist (#8517)

Description

The problem is described by new unit test: t/unit/app/test_app.py::test_config_form_object__module_attr_does_not_exist

When you provide a string with valid module name but invalid submodule (attr) to app.config_from_object the 3rd party util kombu.utils.imports.symbol_by_name would raise AttributeError which would be "handled" by PendingConfiguration(UserDict) and would cause infinite recursion. This happens during the "lazy initialization" of the app.conf when app.conf attr is accessed.

The fix is to catch AttributeError and raise ModuleNotFoundError which would be propagated correctly and would provide the real error message to the end user.

I don't feel very good re: this fix, I am new to this project, so your criticism is highly desired.

Copy link

codecov bot commented Nov 7, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (de0607a) 87.32% compared to head (cf399b7) 87.33%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8619      +/-   ##
==========================================
+ Coverage   87.32%   87.33%   +0.01%     
==========================================
  Files         148      148              
  Lines       18512    18515       +3     
  Branches     3163     3163              
==========================================
+ Hits        16165    16170       +5     
+ Misses       2062     2060       -2     
  Partials      285      285              
Flag Coverage Δ
unittests 87.30% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
celery/app/base.py 96.77% <100.00%> (+0.01%) ⬆️

... and 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Nusnus Nusnus self-requested a review November 7, 2023 16:41
Copy link
Member

@Nusnus Nusnus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!
Check out my comment

with pytest.raises(ModuleNotFoundError) as exc:
self.app.config_from_object(f'{__name__}.bar')
# the module must exist, but it should not have the config attr
assert self.app.conf.broker_url is None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this and the next assertion be outside the with scope?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this resolved? @frolenkov-nikita

celery/app/base.py Outdated Show resolved Hide resolved
celery/app/base.py Outdated Show resolved Hide resolved
@frolenkov-nikita
Copy link
Contributor Author

Thank you very much, I've incorporated your feedback.

@auvipy auvipy added this to the 5.3.x milestone Nov 8, 2023
Copy link
Member

@auvipy auvipy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems better now. waiting for another round of review from Nusnus

@auvipy auvipy merged commit 3723009 into celery:main Nov 8, 2023
32 checks passed
Copy link
Member

@Nusnus Nusnus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay, good work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants