New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-34356: Fix inconsitent behavior of custom formatters in logging config files #8758

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
4 participants
@xavierhardy

xavierhardy commented Aug 13, 2018

add the support for args and kwargs in logging configuration files

https://bugs.python.org/issue34356

@the-knights-who-say-ni

This comment has been minimized.

Show comment
Hide comment
@the-knights-who-say-ni

the-knights-who-say-ni Aug 13, 2018

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately our records indicate you have not signed the CLA. For legal reasons we need you to sign this before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

When your account is ready, please add a comment in this pull request
and a Python core developer will remove the CLA not signed label
to make the bot check again.

You can check yourself
to see if the CLA has been received.

Thanks again for your contribution, we look forward to reviewing it!

the-knights-who-say-ni commented Aug 13, 2018

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately our records indicate you have not signed the CLA. For legal reasons we need you to sign this before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

When your account is ready, please add a comment in this pull request
and a Python core developer will remove the CLA not signed label
to make the bot check again.

You can check yourself
to see if the CLA has been received.

Thanks again for your contribution, we look forward to reviewing it!

fix inconsitent behavior of custom formatters in logging config files
add the support for args and kwargs in logging configuration files
@vsajip

This comment has been minimized.

Show comment
Hide comment
@vsajip

vsajip Aug 19, 2018

Member

Closing, as the issue has been rejected.

Member

vsajip commented Aug 19, 2018

Closing, as the issue has been rejected.

@vsajip vsajip closed this Aug 19, 2018

@xavierhardy

This comment has been minimized.

Show comment
Hide comment
@xavierhardy

xavierhardy Aug 19, 2018

The logging.conf file in INI format is used by OpenStack, that's actually why I submitting this pull request.

https://docs.openstack.org/ocata/config-reference/identity/samples/logging.conf.html

Here is an example with Keystone, but other services are using it as well, such as Mistral in our case. I had to create a handler that actually selected a specific formatter and passed args, kwargs attributes to it, to make it configurable through the logging.conf file.

[loggers]
keys=root

[handlers]
keys=custom

[formatters]
keys=

[logger_root]
level=WARNING
handlers=custom

# using a custom handler that includes its own formatter
[handler_custom]
class=CustomStreamHandler
level=INFO
args=(sys.stdout,'some formatter stuff')

instead of simply doing this:

[loggers]
keys=root

[handlers]
keys=console

[formatters]
keys=custom

[logger_root]
level=WARNING
handlers=console

# using a standard handler
[handler_console]
class=StreamHandler
level=INFO
formatter=custom
args=(sys.stdout,)

# leaving formatting to the formatter
[formatter_custom]
format=(%(name)s): %(asctime)s %(levelname)s %(message)s
class=my.custom.formatter.MyCustomFormatter
args=('some formatter stuff',)

The current implementation is also inconsistent with the way dictConfig handles formatters (it supports both keyword and non-keyword arguments).
https://github.com/python/cpython/blob/master/Lib/logging/config.py#L648

FYI, I've also signed a CLA.

xavierhardy commented Aug 19, 2018

The logging.conf file in INI format is used by OpenStack, that's actually why I submitting this pull request.

https://docs.openstack.org/ocata/config-reference/identity/samples/logging.conf.html

Here is an example with Keystone, but other services are using it as well, such as Mistral in our case. I had to create a handler that actually selected a specific formatter and passed args, kwargs attributes to it, to make it configurable through the logging.conf file.

[loggers]
keys=root

[handlers]
keys=custom

[formatters]
keys=

[logger_root]
level=WARNING
handlers=custom

# using a custom handler that includes its own formatter
[handler_custom]
class=CustomStreamHandler
level=INFO
args=(sys.stdout,'some formatter stuff')

instead of simply doing this:

[loggers]
keys=root

[handlers]
keys=console

[formatters]
keys=custom

[logger_root]
level=WARNING
handlers=console

# using a standard handler
[handler_console]
class=StreamHandler
level=INFO
formatter=custom
args=(sys.stdout,)

# leaving formatting to the formatter
[formatter_custom]
format=(%(name)s): %(asctime)s %(levelname)s %(message)s
class=my.custom.formatter.MyCustomFormatter
args=('some formatter stuff',)

The current implementation is also inconsistent with the way dictConfig handles formatters (it supports both keyword and non-keyword arguments).
https://github.com/python/cpython/blob/master/Lib/logging/config.py#L648

FYI, I've also signed a CLA.

@vsajip

This comment has been minimized.

Show comment
Hide comment
@vsajip

vsajip Aug 20, 2018

Member

The current implementation is also inconsistent

Just to be clear, do you mean that dictConfig doesn't support arbitrary keywords in formatter configurations? For custom usage, you can set up your own factory using the () key.

I would strongly urge that fileConfig usage moves to dictConfig, which is why I didn't want to add any features to fileConfig, particularly ones that use eval(). Are you a member of the OpenStack project, or just a user? If the latter, I suggest raising an issue there and suggesting to them that fileConfig has been superseded and that usage of it would best be phased out.

I note that you do have a workaround in the form of a custom handler.

Wouldn't this change lead to config files which some versions of Python would silently potentially fail on?

Member

vsajip commented Aug 20, 2018

The current implementation is also inconsistent

Just to be clear, do you mean that dictConfig doesn't support arbitrary keywords in formatter configurations? For custom usage, you can set up your own factory using the () key.

I would strongly urge that fileConfig usage moves to dictConfig, which is why I didn't want to add any features to fileConfig, particularly ones that use eval(). Are you a member of the OpenStack project, or just a user? If the latter, I suggest raising an issue there and suggesting to them that fileConfig has been superseded and that usage of it would best be phased out.

I note that you do have a workaround in the form of a custom handler.

Wouldn't this change lead to config files which some versions of Python would silently potentially fail on?

@xavierhardy

This comment has been minimized.

Show comment
Hide comment
@xavierhardy

xavierhardy Aug 21, 2018

dictConfig does support them.

I'm a minor contributor to the OpenStack project and an OpenStack user as well. I contributed on Mistral (workflow orchestration).

You're right for eval(), it's true it's not a good idea, but since it was already there, I decided to go for it. I'll try to check later this week if there's already a ticket concerning the support of dictConfig in OpenStack. I think the best would be to support both for retro-compatibility.

Thanks for your help.

xavierhardy commented Aug 21, 2018

dictConfig does support them.

I'm a minor contributor to the OpenStack project and an OpenStack user as well. I contributed on Mistral (workflow orchestration).

You're right for eval(), it's true it's not a good idea, but since it was already there, I decided to go for it. I'll try to check later this week if there's already a ticket concerning the support of dictConfig in OpenStack. I think the best would be to support both for retro-compatibility.

Thanks for your help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment