New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set default logger in logging Mixin #20355
Conversation
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
Seems like this one detected an interestig error - some of the classes using Logging Mixin do not call super.init() - and cause our tests to fail |
703e6a1
to
f6b6f3d
Compare
I just added one more check and seems ok |
f6b6f3d
to
d4808a2
Compare
I need a second committer approval on that one |
I found the commit when this comment was added. |
d4808a2
to
a5b8b33
Compare
airflow/utils/log/logging_mixin.py
Outdated
# FIXME: LoggingMixin should have a default _log field. | ||
return self._log # type: ignore | ||
except AttributeError: | ||
if not hasattr(self, '_log') or self._log is None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why is the hasattr()
check needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some of the classes that inherit from LoggingMixin don't call super().__init__()
.
I counted above 5 of them.
For that cases hasattr
is needed.
I assume that or self._log is None
could be deleted.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep. The logging Mixin is not consistently used. I think it's safer to leave it here I think as it's rather likely there will be other cases where init() is not called for the Mixin.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How about making _log
a class variable instead?
class LoggingMixin:
_log: Optional[logging.Logger] = None
def __init__(self, context=None):
self._set_context(context)
@property
def log(self) -> Logger:
"""Returns a logger."""
if self._log is None:
self._log = ...
return self._log
I think this should work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
The advantage of this solution will be the absence of usage hasattr
function and all child classes will have _log
attribute.
Firstly I thought that if the child class doesn't use .log
then _log
attribute just will allocate memory useless. But then I realized that I don't see a reason to inherit from LoggingMixin
and not use .log
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Class attributes are not allocated for each subclass, only once for the parent class that defines it, so subclasses would have no additional allocations to choose waste, even if they do inherit LoggingMixin without using log
.
a5b8b33
to
68af0d9
Compare
68af0d9
to
7a5c164
Compare
As was written in this file
"LoggingMixin should have a default _log field."
So I think there is a time to do this. I will be very glad to any suggestions.
Motivation:
Now overriding the method
__getattr__
in every child of this mixin causes recursion and as a resultRecursionError
.