Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make logging.yml read by default #3831

Merged
merged 21 commits into from
May 8, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/source/logging/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,19 @@ After setting the environment variable, any subsequent Kedro commands use the lo
If the `KEDRO_LOGGING_CONFIG` environment variable is not set, Kedro will use the [default logging configuration](https://github.com/kedro-org/kedro/blob/main/kedro/framework/project/default_logging.yml).
```

## Custom `CONF_SOURCE` with logging
SajidAlamQB marked this conversation as resolved.
Show resolved Hide resolved

When you customise the [`CONF_SOURCE`](../configuration/configuration_basics.md#how-to-change-the-configuration-source-folder-at-runtime) setting in your Kedro project, it determines where Kedro looks for configuration files, it does not automatically affect where the `logging.yml` is searched for.

By default, Kedro expects the logging configuration file to be located in the `conf` directory. If you change `CONF_SOURCE` and want to use a different location for your `logging.yml`, you must update the `KEDRO_LOGGING_CONFIG` environment variable to point to the new location of your logging configuration.

For example:
```bash
export KEDRO_LOGGING_CONFIG=<project_root>/custom_config_folder/logging.yml
```

Please note that adjusting `CONF_SOURCE` without updating the logging configuration accordingly can lead to Kedro not locating the `logging.yml` file, which will result in the default logging settings being used instead.

### How to show DEBUG level messages
To see `DEBUG` level messages, change the level of logging in your project-specific logging configuration file (`logging.yml`). We provide a `logging.yml` template:

Expand Down
21 changes: 17 additions & 4 deletions kedro/framework/project/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -216,10 +216,23 @@ def configure(self, pipelines_module: str | None = None) -> None:
class _ProjectLogging(UserDict):
def __init__(self) -> None:
"""Initialise project logging. The path to logging configuration is given in
environment variable KEDRO_LOGGING_CONFIG (defaults to default_logging.yml)."""
path = os.environ.get(
"KEDRO_LOGGING_CONFIG", Path(__file__).parent / "default_logging.yml"
)
environment variable KEDRO_LOGGING_CONFIG (defaults to conf/logging.yml)."""

# Check if a user path is set in the environment variable
user_logging_path = os.environ.get("KEDRO_LOGGING_CONFIG")

# Check if the default logging configuration exists
default_logging_path = Path("conf/logging.yml")
SajidAlamQB marked this conversation as resolved.
Show resolved Hide resolved
if not default_logging_path.exists():
default_logging_path = Path(__file__).parent / "default_logging.yml"

# Use the user path if available, otherwise, use the default path
if user_logging_path and Path(user_logging_path).exists():
path = Path(user_logging_path)
else:
path = default_logging_path

# Load and apply the logging configuration
logging_config = Path(path).read_text(encoding="utf-8")
self.configure(yaml.safe_load(logging_config))

Expand Down
Loading