Decouple logging.yml from ConfigLoader
#2637
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
…onal targets (#2557) Ready at last. Still some conversation to be had about how we reflect "confidence" in the plugins we describe (what version they were last tested against) but the text is ready and better than what's there currently, so let's go!
#2568) * Stop producing `.egg` files from `kedro package` Fix gh-2273. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Upgrade import-linter Fix gh-2570. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Partially remove reliance on setup.py for micropackaging Incomplete effort, see gh-2414. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Revert "Partially remove reliance on setup.py for micropackaging" This reverts commit fd3efa8. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
…ntime_params properly (#2467) * Fix typehint Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * test push Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * POC of fix to solve the runtime param resolution problem Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix OmegaConfigLoadaer - resolve runtime_params early Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Delegate the intialization of runtime_params to AbstractConfigLoader Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add test for interpolated value and globals Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * add more test and linting Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * refactor Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * update release note Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Apply comments and refactor the test Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update RELEASE.md Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com> --------- Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
* Partially revert #2378 as OCL no longer returning DictConfig Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Revert #2176 with special logging handling Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * remove unused import Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
…ibution section (#2579) * modify databricks testing guide for contributors Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/contribution/development_for_databricks.md Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com> * Update development_for_databricks.md --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jannic <37243923+jmholzer@users.noreply.github.com>
… file is empty (#2584) * Allow empty config files Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Remove condition for empty config Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Revert "Remove condition for empty config" This reverts commit 7961155. * Update release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add unit test Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
* Proposal of updated settings.py Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Update settings after comment Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
* Add metadata attribute to kedro.io datasets Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
* Update pipeline hooks docstrings Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com> * Acknowledge community contribution --------- Signed-off-by: Tomas Van Pottelbergh <github@tomasvanpottelbergh.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Add debug level logging for hook execution order Fix gh-1942. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Clarify role of setuptools in hooks Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Remove mentions to outdated and unused `pkg_resources` Fix gh-2391. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Adapt example hook to spec signature Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Grammar and style fixes in docs Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Revert "Adapt example hook to spec signature" This reverts commit ac1b495. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
…d" with "legacy" (#2606)
* FIX: Typo in markdown for kedro notebooks docs Signed-off-by: debugger24 <rahulcomp24@gmail.com> * Acknowledge community contribution --------- Signed-off-by: debugger24 <rahulcomp24@gmail.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Add the Kedro RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Clean up imports and comments Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Map arguments to add flexibility to customize arguments available in RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Clean up rich_logger.py and unused imports Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add docs - first iteration Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix imports and small refactoring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Improve the doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update the project's template Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * format docstring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Apply Jo's comment on the doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix the link with RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Apply suggestions from code review Co-authored-by: Antony Milne <49395058+antonymilne@users.noreply.github.com> * Fix broken doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * update template defaults Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * refactoring RichHandler and improve doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix broken test due to new default RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Move kedro.extras.logging.RichHandler -> kedro.logging.RichHandler Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * add kedro/logging.py and remove the unnecessary import that cause namespace collision Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Refactor test Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add more tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix lots of import, docstring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Reorder docstring Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Link options to doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Clean up doc and unused logger Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix spelling `customise` Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Refactor test with fixture Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Remove unnecessary copy() because it is now a fixture Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Split up test_rich_traceback_configuration() to smaller tests Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add RELEASE.md Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update doc Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix template logging
* Mention awesome-kedro in docs See kedro-org/kedro-devrel#40 * Update CONTRIBUTING.md * Spelling and style fixes Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
* Update SnowparkTableDataSet docs * Add missing snowpark dependencies for docs building Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Update __init__.py * Update release note Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Update Version Numbers across docs Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Update RELEASE.md * Update Release Note Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
…#2595) * Add deployment workflow page Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add table of contents, entry point guide, data and conf upload guide Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add detailed instructions for creating a job on Databricks Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add images and automated deployment resources Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Remove use of 'allows', add summary Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Remove link to missing image Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add deployment workflow to toctree Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Lint and fix missing link Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Minor style, syntax and grammar improvements Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Fixes for correctness during validation Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add instructions for creating log output location Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Lint Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Lint databricks_run Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Minor wording change in reference to logs Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Modify reference to Pyspark-Iris Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix linter errors to enable docs build for inspection Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update build-docs.sh * Fix broken link Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove spurious word Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add advantages subheading Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update docs/source/integrations/databricks_deployment_workflow.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add alternative ways to upload data to DBFS Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Move note on unpackaged config and data Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Fix broken links Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Move databricks back into deployment section Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove references to PySpark Iris (pyspark-iris) starter Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Graphics links fixes, revise titles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix broken internal link Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Fix links broken by new folder Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove logs directory Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Update image of final job configuration Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Add full stops in list. Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Fix conda environment name. Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Modify wording and image for creating a new job cluster Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Modify wording in guide to create new job cluster Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> * Remove --upgrade option Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Add both ways of creating a new job Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> --------- Signed-off-by: Jannic Holzer <jannic.holzer@quantumblack.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
* Update docs font family to Inter Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Match weights to previous Google font Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update logo images; revert css formatting for easier PR review Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update text styles Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Add trailing whitespace (?) Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update Kedro image path Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update font size Signed-off-by: Tynan DeBold <thdebold@gmail.com> * update font to font-family: 'Inter', sans-serif; Signed-off-by: huongg <huongg1409@gmail.com> * testing with important for font-family Signed-off-by: huongg <huongg1409@gmail.com> * Make project name lowercase in docs Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * revert back to kedro/develop Signed-off-by: huongg <huongg1409@gmail.com> --------- Signed-off-by: Tynan DeBold <thdebold@gmail.com> Signed-off-by: huongg <huongg1409@gmail.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com> Co-authored-by: huongg <huongg1409@gmail.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
#2621) * Enable variable interpolation in catalog by escaping _ Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com>
Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
5 tasks
|
Replace by #2642, This was created from |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Fix #2426
Following #2281 we have decided that users will no longer be able to change logging configuration dependent on kedro run environment. This is a huge simplification and means that logging only gets configured once in the kedro process with
LOGGING = _ProjectLogging().There's a few things to be done here, but they all need to be done at the same time or it will be very confusing.
KedroSession._setup_loggingandKedroSession._get_logging_config🥳config_patternsand docstring examples)loggingthroughout the whole codebase. Note thatconfigure_loggingmust stay since it's needed by the parallel runnerconf/base/logging.ymlKEDRO_LOGGING_CONFIGis set to point to itlogging.ymlfile that a user could download and put in theirconfdirectory if they wanted to, e.g. to add file-based logging. Note that after this change, file-based logging becomes opt-in rather than opt-out as it is currently. Docs should reflect this.Development notes
Checklist
RELEASE.mdfile