Skip to content

Commit

Permalink
Warnings controlled by logger level (huggingface#26527)
Browse files Browse the repository at this point in the history
* Logger level

Co-authored-by: Sahil Bhosale <sahilbhosale63@live.com>
Co-authored-by: Adithya4720 <hegdeadithyak@gmail.com>
Co-authored-by: Sachin Singh <sachinishu02@gmail.com>
Co-authored-by: Riya Dhanduke <113622644+riiyaa24@users.noreply.github.com>

* More comprehensive documentation

---------

Co-authored-by: Sahil Bhosale <sahilbhosale63@live.com>
Co-authored-by: Adithya4720 <hegdeadithyak@gmail.com>
Co-authored-by: Sachin Singh <sachinishu02@gmail.com>
Co-authored-by: Riya Dhanduke <113622644+riiyaa24@users.noreply.github.com>
  • Loading branch information
5 people authored and helboukkouri committed Oct 16, 2023
1 parent bdbaccc commit 7932074
Show file tree
Hide file tree
Showing 2 changed files with 43 additions and 0 deletions.
17 changes: 17 additions & 0 deletions docs/source/en/main_classes/logging.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,23 @@ verbose to the most verbose), those levels (with their corresponding int values

By default, `tqdm` progress bars will be displayed during model download. [`logging.disable_progress_bar`] and [`logging.enable_progress_bar`] can be used to suppress or unsuppress this behavior.

## `logging` vs `warnings`

Python has two logging systems that are often used in conjunction: `logging`, which is explained above, and `warnings`,
which allows further classification of warnings in specific buckets, e.g., `FutureWarning` for a feature or path
that has already been deprecated and `DeprecationWarning` to indicate an upcoming deprecation.

We use both in the `transformers` library. We leverage and adapt `logging`'s `captureWarning` method to allow
management of these warning messages by the verbosity setters above.

What does that mean for developers of the library? We should respect the following heuristic:
- `warnings` should be favored for developers of the library and libraries dependent on `transformers`
- `logging` should be used for end-users of the library using it in every-day projects

See reference of the `captureWarnings` method below.

[[autodoc]] logging.captureWarnings

## Base setters

[[autodoc]] logging.set_verbosity_error
Expand Down
26 changes: 26 additions & 0 deletions src/transformers/utils/logging.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@
WARN, # NOQA
WARNING, # NOQA
)
from logging import (
captureWarnings as _captureWarnings,
)
from typing import Optional

import huggingface_hub.utils as hf_hub_utils
Expand Down Expand Up @@ -121,6 +124,29 @@ def get_log_levels_dict():
return log_levels


def captureWarnings(capture):
"""
Calls the `captureWarnings` method from the logging library to enable management of the warnings emitted by the
`warnings` library.
Read more about this method here:
https://docs.python.org/3/library/logging.html#integration-with-the-warnings-module
All warnings will be logged through the `py.warnings` logger.
Careful: this method also adds a handler to this logger if it does not already have one, and updates the logging
level of that logger to the library's root logger.
"""
logger = get_logger("py.warnings")

if not logger.handlers:
logger.addHandler(_default_handler)

logger.setLevel(_get_library_root_logger().level)

_captureWarnings(capture)


def get_logger(name: Optional[str] = None) -> logging.Logger:
"""
Return a logger with the specified name.
Expand Down

0 comments on commit 7932074

Please sign in to comment.