"Logging HOWTO" should share an example of best practices for using logging in a library

[nathanielmanistaatgoogle]

BPO	34590
Nosy	@rhettinger, @vsajip, @nathanielmanistaatgoogle

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/vsajip'
closed_at = <Date 2018-10-17.01:42:04.738>
created_at = <Date 2018-09-05.20:39:26.348>
labels = ['3.8', 'type-feature', '3.7', 'invalid', 'docs']
title = '"Logging HOWTO" should share an example of best practices for using logging in a library'
updated_at = <Date 2018-10-17.01:42:04.737>
user = 'https://github.com/nathanielmanistaatgoogle'

bugs.python.org fields:

activity = <Date 2018-10-17.01:42:04.737>
actor = 'vinay.sajip'
assignee = 'vinay.sajip'
closed = True
closed_date = <Date 2018-10-17.01:42:04.738>
closer = 'vinay.sajip'
components = ['Documentation']
creation = <Date 2018-09-05.20:39:26.348>
creator = 'Nathaniel Manista'
dependencies = []
files = []
hgrepos = []
issue_num = 34590
keywords = []
message_count = 7.0
messages = ['324652', '324660', '324711', '324726', '324727', '324745', '327850']
nosy_count = 6.0
nosy_names = ['rhettinger', 'vinay.sajip', 'tlesher', 'docs@python', 'Nathaniel Manista', 'pms.coder']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue34590'
versions = ['Python 3.7', 'Python 3.8']

[nathanielmanistaatgoogle]

https://docs.python.org/3.8/howto/logging.html#configuring-logging-for-a-library is a bit too short and doesn't answer some questions that I have as a library author about the best ways to use logging in a library.

Should I make use of a single logger object in my library, multiple loggers in a tree, or multiple unrelated loggers? Since I have just one public module, I'm tempted to say that I should just use one logger object.

Should I present as part of my public API the name of the logger object(s) used? Probably - the documentation starts with "When developing a library which uses logging, you should take care to document how the library uses logging - for example, the names of loggers used.". But should I also include the logger objects in my API? If an application wants to reach my logger, why should they use "logging.getLogger(<name that I shared with them>)" rather than "my_library.LOGGER"?

Should I use my library's fully-qualified name as the name of its logger? This seems like a great idea because it's unlikely to bring up any new conflicts. Is it a best practice? Is there any better practice?

The "Configuring Logging for a Library" text could answer these questions, and maybe it should, but... I really think a toy library example that library authors could follow would also help a great deal.

[rhettinger]

FWIW, I think the practices vary somewhat widely and that none of your questions have generally agreed upon answers. Part of the reason that there are so many ways to go is that the package was modeled after Java APIs where the practices and needs were also widely varied. Given the absence of clear-cut best practices, the docs should probably defer to StackOverflow and various blog posts.

[nathanielmanistaatgoogle]

Something... related, that may perhaps belong in a separate issue, but that I want to at least mention here because it would be solved if logging-in-libraries best practices were authoritatively documented and exemplified: it's just too consarn easy to "hold [the logging module] wrong" and wind up with "No handlers could be found for logger" log spam: https://www.google.ca/search?q=site:stackoverflow.com+"No+handlers+could+be+found+for+logger". Why should a Logger object need to have .basicConfig() called on it after retrieval (where "retrieval" means "getLogger call") and before use anyway? Why can't it just be ready for use (at least as ready for use as .basicConfig makes it) when passed from the logging module to the logging-using module like nearly any other object passed from the standard library to standard-library-using code?

(The documentation says "No handlers could be found for logger" spam only happens pre-3.2, but some of my users at least think they see it in later Pythons.)

[pmscoder]

I would be happy to see such HOWTO, so we could relay on actual documentation instead of the "Internet".

I think the best approach would be to get those 10-20 questions you ( Nathaniel Manista) just asked and post them to python-dev mailing list, so senior developers/committers could answer them.

Then we could establish the "gold standard" for logging and write about it in the HOWTO.

[rhettinger]

Then we could establish the "gold standard" for logging and write
about it in the HOWTO.

There really isn't a gold standard. There are many widely varied logging practices, each adapted to the needs of the application. The module is complicated primarily because there are so many legitimate techniques and reasons for using those techniques.

(The documentation says "No handlers could be found for logger"
spam only happens pre-3.2, but some of my users at least think
they see it in later Pythons.)

If you find an actual error in the docs, please make a specific report so that it can be confirmed and fixed.

[vsajip]

Should I make use of a single logger object in my library, multiple loggers
in a tree, or multiple unrelated loggers? Since I have just one public module,
I'm tempted to say that I should just use one logger object.

Yes. The advanced logging tutorial (part of the HOWTO) states:

"A good convention to use when naming loggers is to use a module-level logger, in each module which uses logging, named as follows:

logger = logging.getLogger(__name__)

This means that logger names track the package/module hierarchy, and it’s intuitively obvious where events are logged just from the logger name."

This seems clear enough to me. Did you look at the whole of the HOWTO? If not, I'd advise you to take the time

(The documentation says "No handlers could be found for logger"
spam only happens pre-3.2, but some of my users at least think
they see it in later Pythons.)

In 3.2, the change was made to introduce a "handler of last resort" instead of the "no handlers could be found" message, which is only printed if logging.lastResort is set to None or other "false" value (by default, it's set to a console handler at WARNING level emitting to sys.stderr). If you can give specifics of a later Python version where this happens (precise Python version, OS etc.) then we can address it.

Practices vary widely because use cases for logging are widely varied. A simple one-off script has different needs from a product with a long shelf-life, a command-line application has different needs from a long-running server application, a single-author application has different needs from one which is put together from lots of different dependencies by different authors, etc.

Why should a Logger object need to have .basicConfig() called on
it after retrieval (where "retrieval" means "getLogger call") and
before use anyway?

It doesn't. If you want output, you need handlers. The basicConfig() API is just one way of attaching handlers to loggers for simple use cases. The question of handlers is also no concern of a library author, other than the NullHandler part, as the application is where logging should be configured (as mentioned in the "Configuring Logging for a Library" section).

Then we could establish the "gold standard" for logging and write
about it in the HOWTO.

There's no one size fits all, I'm afraid, because the package is designed to cover a wide range of use cases, not just the simplest ones. While there is (IMO) comprehensive documentation, specific proposals for improvements will be looked at constructively.

[vsajip]

Closing, as no specific proposals received.