Add new CLI commands to dataset factory docs #2930
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
AhdraMeraliQB
requested review from
yetudada,
astrojuanlu and
stichbury
as code owners
ankatiyar
reviewed
Aug 16, 2023
ankatiyar
reviewed
Aug 16, 2023
| By default this is output to the terminal. However, if you wish to output the resolved catalog to a specific file, you can use the redirection operator `>`: | ||
|
|
||
| ```bash | ||
| kedro catalog resolve > output_file.yaml |
There was a problem hiding this comment.
Suggested change
| kedro catalog resolve > output_file.yaml | |
| kedro catalog resolve > conf/base/catalog_resolved.yaml |
There was a problem hiding this comment.
This can then be picked up by the config loader, but then the pre-existing datasets that came from explicit entries in the catalog would need to be commented out so there's no duplicates. Maybe we could suggest this in the docs? Not sure if this would make this part of the docs unnecessarily complicated for the users though so feel free to ignore this.
There was a problem hiding this comment.
I worry this might be a bit out of place, the beginning of this page in the docs does already talk about having different catalog files in different environments, it might get a bit messy to start explaining that here as well? I'm leaning against it but also curious if anyone else has any other opinions on this
docs/source/data/data_catalog.md
Outdated
|
|
||
| #### Using `kedro catalog rank` | ||
|
|
||
| This command outputs a list of all dataset factories in the catalog, ranked by the priority that they are matched against. This priority is determined by considering the following criteria: |
There was a problem hiding this comment.
Idk if "priority" is the right word. Something like "ranked according to the order in which they are matched with dataset names mentioned in your pipelines"?
There was a problem hiding this comment.
Added in 2803702
| Consider a catalog file with the following patterns: | ||
|
|
||
| ```yaml | ||
| "{layer}.{dataset_name}": |
There was a problem hiding this comment.
I would also add an example where the number of characters outside the placeholders is same for two patterns but one is ranked higher because of the number of placeholders. For eg. {whatever}_{dataset}csv would be ranked higher than {dataset}#csv for preprocessed_companies#csv.
There was a problem hiding this comment.
Added in 569cd63
| func=create_model_input_table, | ||
| inputs=["preprocessed_shuttles", "preprocessed_companies", "reviews"], | ||
| outputs="model_input_table", | ||
| name="create_model_input_table_node", |
There was a problem hiding this comment.
Maybe adding some datasets in the pipeline that would match the catch-all pattern would also be nice to demonstrate here - Datasets that would normally be MemoryDataSets
There was a problem hiding this comment.
model_input_table matches the catch-all pattern
There was a problem hiding this comment.
Ah, my bad. Maybe adding an explanation about which factory pattern resolved which dataset and which ones came from the catalog will be helpful for users?
merelcht
reviewed
Aug 16, 2023
There was a problem hiding this comment.
I left some comments about the "rank" command. I think it's better to be overly explicit when explaining how the command works. In the user research a lot of people mentioned they thought it was important to have very descriptive docs.
@stichbury has an open PR moving some of the catalog docs around, so you two should probably coordinate who merges first.
docs/source/data/data_catalog.md
Outdated
Comment on lines
705
to
708
| - preprocessed_{dataset_name} | ||
| - processed_{dataset_name} | ||
| - '{layer}.{dataset_name}' | ||
| - '{default_dataset}' |
There was a problem hiding this comment.
I think it might be good to add why each of these has this rank, e.g. preprocessed_{dataset_name} has the most non-placeholder characters etc.
There was a problem hiding this comment.
Added in 5200ca5
Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
AhdraMeraliQB
added a commit
that referenced
this pull request
Aug 17, 2023
Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com>
|
Closing in favour of #2935 |
AhdraMeraliQB
added a commit
that referenced
this pull request
Aug 17, 2023
* Add changes from #2930 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Lint Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Make code snippets collapsable Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> --------- Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
stichbury
added a commit
that referenced
this pull request
Aug 18, 2023
* First drop of newly organised data catalog docs Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * linter Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Added to-do notes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Afternoon's work in rewriting/reorganising content Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * More changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Further changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Another chunk of changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Final changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Revise ordering of pages Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add new CLI commands to dataset factory docs (#2935) * Add changes from #2930 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Lint Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Make code snippets collapsable Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> --------- Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Bunch of changes from feedback Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * A few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update h1,h2,h3 font sizes Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Add code snippet for using DataCatalog with Kedro config Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/data/data_catalog.md * Upgrade kedro-datasets for docs Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Improve prose Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Signed-off-by: Tynan DeBold <thdebold@gmail.com> Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Co-authored-by: Tynan DeBold <thdebold@gmail.com> Co-authored-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
ankatiyar
added a commit
that referenced
this pull request
Sep 12, 2023
* Remove non-user-facing release notes about pip pin (#2854) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Allow for customization of pipeline templates Controlled via `settings.PIPELINE_TEMPLATE_PATH`. Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update lookup based on feedback Instead of lookups on `settings.py` just statically look for a `templates` directory in the project root. If this folder is present it will override the default global template, otherwise fall back to the previous global default. Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add CLI flag for pipeline creation template -t/--template Precedence for template_path is: command line > project templates/pipeline dir > global default pipeline create now will also print the template folder used. Signed-off-by: Jason Hite <jasonmhite@gmail.com> * First basic unit test TODO is testing the CLI flag too, and maybe testing that the cleanup works with custom templates. Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add test to check overriding template path on the CLI Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Linter formatting fixes Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Release 0.18.12 (#2871) * bump version and prepare for new release Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update RELEASE.md Added community contributions * add DeltaTableDataSet to rst Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * temporarily use the fixed kedro-plugins branch Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * test docsbuild on merels branch Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * use kedro-datasets new release Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> --------- Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Fix broken link to Prefect website in deploy guide (#2885) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Clarify the <micropkg_name> argument to kedro micropkg package (#2835) Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Allow registering of custom resolvers to `OmegaConfigLoader` (#2869) * Allow registering of custom resolvers to OCL Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Complete doc string Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add test for overwritten resolvers Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update test for overwritten resolvers Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Remove replace=True by default Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add debug level log for registering new resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Document the use of custom resolvers with `OmegaConfigLoader` (#2896) * Update docs for custom resolvers for OCL Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update FAQ with correct link Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Fix typo Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Add note for oc.env Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Suggestions from code review Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update kedro pipeline create to use new /conf file structure (#2856) * add parameters_pipelineNane.yml template in the conf folder Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * del yml template from parameters folder Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * modify delete pipeline command to use correct path to yml files Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * modify test_pipeline move parameters foldername to the yml filename Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * collect configs to package not only from parameters folder, but from core conf folder also Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * move parameters foldername to the yml filename, remove nesting Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * move parameters foldername to the yml filename, remove nesting Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Add changes to RELEASE.md Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Update RELEASE.md Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Update RELEASE.md, change to release 0.18.13 Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * save old structure for 'pipeline delete' command for backward compatibility Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * modify test_pipeline for 'pipeline delete' backward compability Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update kedro catalog create to use new /conf structure (#2884) * modify test_catalog, remove nesting in the catalog folder Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * move catalog foldername to the yml filename, remove nesting Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Update RELEASE.md Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Modify docs, fix broken link Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add migration steps for `ConfigLoader` to `OmegaConfigLoader` (#2887) * Add migration steps for CL to OCL Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Fix #2498 Adding logging issue 2498 (#2842) * Fixing typo n on line 114 file /docs/source/nodes_and_pipelines/run_a_pipeline.md from issue #2607 Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> * Removed redundant packages at lines 66, 67 and 68 on file kedro/docs/source/nodes_and_pipelines/run_a_pipeline.md, packages: from kedro.io import AbstractDataSet, DataCatalog, MemoryDataSet, from kedro.pipeline import Pipeline and from kedro.runner.runner import AbstractRunner Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> * Add logger object for logging when computing micropackage version Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> * Added new format to the logging Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> --------- Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Try only trigger docs build when release notes updated (#2907) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add Python 3.11 support to Kedro (#2851) * add 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * limit number of python version tests Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * fix setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * delta-spark version Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * try original setup Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove 3.11 from circleci Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin matplotlib to 3.7.2 for python 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update test_import.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add noks test adjustments Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * replace eq with is for multi_catalog Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * minor fixes Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove matplotlib 3.11 pin Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * matplotlib pin for 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * run original matplotlib pin with ubuntu only Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * run circleci Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin spark < 3.4 as its breaking Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * spark datasets revert earlier changes back to original Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * delta Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove all windows tests from GHA Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * skip coverage as not needed Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Checking only WIN tests (will revert) Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update unit-tests.yml Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * undo previous commit Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add msbuild Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add ubuntu tests back in and remove 3.11 tests from circleci add in msbuild for the other tests Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * set hdf5 disable check and lint Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin tables for 3.8 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin 3.8 tables Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * lint Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove 3.11 completely from circleci and tidy up Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add 3.11 to pyarrow pin for docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * undo previous commit Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * update rtd to python 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * try 3.10 on rtd Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * fix outdated pylint to ruff Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * changes based on review Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add mock for pyarrow and deltalake for docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * undo previous commit Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update .readthedocs.yml Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update conf.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * changes based on review Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * fix comments Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * test running 3.10 parallel Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Add to release notes and update badges on docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Add migration tips Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> --------- Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Revise FAQs and README (#2909) * Revise FAQs and README Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add back the data layers FAQ as I've no idea where else it fits Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * minor changes from review Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update Generator example (#2858) * update release note Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * update to binary mode when opening file Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Simplify the implementation of Dataset Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * simplify the example - wip Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update the nodes Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * update example Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update example Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Apply suggestions from code review Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * add more instruction to guide user to create the custom dataset Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * update the reference from `pipeline.py` -> `nodes.py` Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Collapse the nodes.py function into a expandable section Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Update nodes.md * Update nodes.md --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * update docs to reflect change to /conf file structure (#2913) * update docs to reflect change to /conf file structure Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * update docs, folder structure diagrams Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Change CONTRIBUTING.md file based on PR #2894 (#2914) Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Move contribution guidelines from CONTRIBUTING.md to the Wiki (#2894) * Add drafts for new contributing.md and wiki page Signed-off-by: lrcouto <laurarccouto@gmail.com> * Add contribution details to wiki page draft Signed-off-by: lrcouto <laurarccouto@gmail.com> * Remove contributing-to-Kedro.md draft from the branch Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix broken link to Prefect website in deploy guide (#2885) Signed-off-by: lrcouto <laurarccouto@gmail.com> * Clarify the <micropkg_name> argument to kedro micropkg package (#2835) Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Allow registering of custom resolvers to `OmegaConfigLoader` (#2869) * Allow registering of custom resolvers to OCL Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Complete doc string Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add test for overwritten resolvers Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update test for overwritten resolvers Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Remove replace=True by default Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add debug level log for registering new resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Some adjustments to ordering of text Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Document the use of custom resolvers with `OmegaConfigLoader` (#2896) * Update docs for custom resolvers for OCL Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update FAQ with correct link Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Fix typo Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Add note for oc.env Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Suggestions from code review Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update kedro pipeline create to use new /conf file structure (#2856) * add parameters_pipelineNane.yml template in the conf folder Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * del yml template from parameters folder Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * modify delete pipeline command to use correct path to yml files Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * modify test_pipeline move parameters foldername to the yml filename Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * collect configs to package not only from parameters folder, but from core conf folder also Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * move parameters foldername to the yml filename, remove nesting Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * move parameters foldername to the yml filename, remove nesting Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Add changes to RELEASE.md Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Update RELEASE.md Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Update RELEASE.md, change to release 0.18.13 Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * save old structure for 'pipeline delete' command for backward compatibility Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * modify test_pipeline for 'pipeline delete' backward compability Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update CONTRIBUTING.md Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix broken link Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update kedro catalog create to use new /conf structure (#2884) * modify test_catalog, remove nesting in the catalog folder Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * move catalog foldername to the yml filename, remove nesting Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Update RELEASE.md Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> * Modify docs, fix broken link Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> --------- Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Add migration steps for `ConfigLoader` to `OmegaConfigLoader` (#2887) * Add migration steps for CL to OCL Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix #2498 Adding logging issue 2498 (#2842) * Fixing typo n on line 114 file /docs/source/nodes_and_pipelines/run_a_pipeline.md from issue #2607 Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> * Removed redundant packages at lines 66, 67 and 68 on file kedro/docs/source/nodes_and_pipelines/run_a_pipeline.md, packages: from kedro.io import AbstractDataSet, DataCatalog, MemoryDataSet, from kedro.pipeline import Pipeline and from kedro.runner.runner import AbstractRunner Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> * Add logger object for logging when computing micropackage version Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> * Added new format to the logging Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> --------- Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Try only trigger docs build when release notes updated (#2907) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Add Python 3.11 support to Kedro (#2851) * add 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * limit number of python version tests Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * fix setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * delta-spark version Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * try original setup Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove 3.11 from circleci Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin matplotlib to 3.7.2 for python 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update test_import.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add noks test adjustments Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * replace eq with is for multi_catalog Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * minor fixes Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove matplotlib 3.11 pin Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * matplotlib pin for 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * run original matplotlib pin with ubuntu only Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * run circleci Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin spark < 3.4 as its breaking Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * spark datasets revert earlier changes back to original Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * delta Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove all windows tests from GHA Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * skip coverage as not needed Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Checking only WIN tests (will revert) Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update unit-tests.yml Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * undo previous commit Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add msbuild Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add ubuntu tests back in and remove 3.11 tests from circleci add in msbuild for the other tests Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * set hdf5 disable check and lint Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin tables for 3.8 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * pin 3.8 tables Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * lint Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * remove 3.11 completely from circleci and tidy up Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add 3.11 to pyarrow pin for docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * undo previous commit Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * update rtd to python 3.11 Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * try 3.10 on rtd Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * fix outdated pylint to ruff Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * changes based on review Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add mock for pyarrow and deltalake for docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * undo previous commit Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update setup.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update .readthedocs.yml Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update conf.py Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * changes based on review Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * fix comments Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * test running 3.10 parallel Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Add to release notes and update badges on docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Add migration tips Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> --------- Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Revise FAQs and README (#2909) * Revise FAQs and README Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add back the data layers FAQ as I've no idea where else it fits Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * minor changes from review Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update Generator example (#2858) * update release note Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * update to binary mode when opening file Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Simplify the implementation of Dataset Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * simplify the example - wip Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update the nodes Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * update example Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Update example Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Apply suggestions from code review Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * add more instruction to guide user to create the custom dataset Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * update the reference from `pipeline.py` -> `nodes.py` Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Collapse the nodes.py function into a expandable section Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Update nodes.md * Update nodes.md --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> * Update CONTRIBUTING.md --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Co-authored-by: Jonathan Cohen <jon@joncohen.dev> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Dmitry Sorokin <40151847+DimedS@users.noreply.github.com> Co-authored-by: Laíza Milena Scheid Parizotto <41296740+laizaparizotto@users.noreply.github.com> Co-authored-by: Sajid Alam <90610031+SajidAlamQB@users.noreply.github.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Remove redundant pages and direct users to wiki (#2917) * Remove redundant pages and direct users to wiki Signed-off-by: lrcouto <laurarccouto@gmail.com> * Lint docs changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Few tweaks to index page Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Deprecate abstract "DataSet" in favor of "Dataset" (#2746) * Deprecate abstract "DataSet" in favor of "Dataset" Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Update docs references to abstract "DataSet" class Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Fix `AbstractDataSet` reference to `AbstractDataset` Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Update docs/source/conf.py Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Change remaining `_DEPRECATED_ERROR_CLASSES` to `_DEPRECATED_CLASSES`, update type hints Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Update RELEASE.md * Reformat kedro/io/__init__.py with Black and isort Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Remove duplicate imports Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Move imports Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> --------- Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Reorganise and improve the data catalog documentation (#2888) * First drop of newly organised data catalog docs Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * linter Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Added to-do notes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Afternoon's work in rewriting/reorganising content Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * More changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Further changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Another chunk of changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Final changes Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Revise ordering of pages Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add new CLI commands to dataset factory docs (#2935) * Add changes from #2930 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Lint Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Make code snippets collapsable Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> --------- Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Bunch of changes from feedback Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * A few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update h1,h2,h3 font sizes Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Add code snippet for using DataCatalog with Kedro config Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/data/data_catalog.md * Upgrade kedro-datasets for docs Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Improve prose Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Signed-off-by: Tynan DeBold <thdebold@gmail.com> Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Ahdra Merali <90615669+AhdraMeraliQB@users.noreply.github.com> Co-authored-by: Tynan DeBold <thdebold@gmail.com> Co-authored-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add line about Viz to PR template (#2929) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add architecture graphic back to docs with revisions (#2916) * Revise FAQs and README Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Add back the data layers FAQ as I've no idea where else it fits Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * minor changes from review Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Re-add kedro arch diagram, with revised graphic Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * fix broken anchor Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * fix broken anchor Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update docs/source/extend_kedro/architecture_overview.md Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> * Update docs/source/extend_kedro/architecture_overview.md Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> * Update docs/source/extend_kedro/architecture_overview.md Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> * Update docs/source/extend_kedro/architecture_overview.md Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> * Changes to architecture page following review Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Change diagram following reivew * Add links to API docs Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Added in type of users * Fix linting error Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Co-authored-by: Yetunde Dada <43755008+yetudada@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add kedro catalog resolve command (#2891) * Implement working draft Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Remove comments Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Remove raw_config Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Use config loader to access catalog config Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Use dict comprehension Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Remove pipeline filtering Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Prevent overwrite or param inclusion Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Trim filepath Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Appease linter Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add test for resolve Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add test to exclude params Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add test for overwrite (not working) Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Fix test Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Remove print Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add changes to RELEASE.md Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Trim unreachable code Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Make helper function private Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Refactor Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add changes to the docs Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Refactor 2 Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Fix docs build Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add suggestions from code review (docs) Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Add suggestions from code review (code) Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Rename catalog_config variable to explicit_datasets Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Apply suggestions from code review Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Fix mocking in test Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Change test fixture name Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> * Lint Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> --------- Signed-off-by: Ahdra Merali <ahdra.merali@quantumblack.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Replace "DataSet" with "Dataset" in Markdown files (#2735) * LambdaDataSet->LambdaDataset in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * MemoryDataSet->MemoryDataset in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * PartitionedDataSet->PartitionedDataset in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * IncrementalDataSet->IncrementalDataset in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * CachedDataSet->CachedDataset in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * DataSetError->DatasetError in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * DataSetNotFoundError->DatasetNotFoundError in .md files Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Replace "DataSet" with "Dataset" in Markdown files * Update RELEASE.md * Fix remaining instance of "*DataSet*"->"*Dataset*" Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * `find . -name '*.md' -print0 | xargs -0 sed -i "" "s/\([^A-Za-z]\)DataSet/\1Dataset/g"` Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Change non-class instances of Dataset to dataset * Replace any remaining instances of DataSet in docs * Fix a broken link to docs for `PartitionedDataset` --------- Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update h1,h2,h3 font sizes in the docs pages (#2938) * Update h1,h2,h3 font sizes Signed-off-by: Tynan DeBold <thdebold@gmail.com> * Update h2 font size Signed-off-by: Tynan DeBold <thdebold@gmail.com> --------- Signed-off-by: Tynan DeBold <thdebold@gmail.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Automatically trigger `kedro-starters` release on the release of `kedro` (#2889) * Add GA for triggering starters release Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update token name Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Create issues_metrics.yml (#2814) * Create issues_metrics.yml * Update issues_metrics.yml Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Clean up setuptools and wheel requirements to align with PEP-518 (#2927) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Clean up `kedro pipeline create` outdated docs (#2945) * remove the README template generate by kedro pipeline create Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * fix docstring for kedro catalog create Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Fix broken link Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * fix broken link Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add globals feature for `OmegaConfigLoader` using a globals resolver (#2921) * Refactor load_and_merge_dir() Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Try adding globals resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Minor change Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add globals resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Revert refactoring Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add test + remove self.globals Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Allow for nested variables in globals Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add documentation Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Typo Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add error message + test Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Apply suggestions from code review Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Split test into multiple tests Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Restrict the globals config_patterns Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Release notes Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update docs/source/configuration/advanced_configuration.md Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> * Add helpful error message for keys starting with _ Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Enable setting default value for globals resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Typo Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Move test for keys starting with _ to the top Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Fix cross ref link in docs Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Co-authored-by: Merel Theisen <49397448+merelcht@users.noreply.github.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Consolidate two `ruff-pre-commit` entries into one (#2881) * Clean up Ruff configuration in base pyproject.toml Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Consolidate two `ruff-pre-commit` entries into one Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Move Ruff hook to the top, because it's aggressive Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Add `kedro` as `known-first-party` for `isort` run Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Add noqa in docs/source/conf.py Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> * Fix ruff linter precommit config Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Fix typos across the documentation (#2956) * Fix typos across docs Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Capitalisation stuff Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Setup Vale linter as a GHA workflow (#2953) * Add vale GHA Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update condition Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update .github/styles/Kedro/ignore.txt Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> * add some styles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Remove styles for now Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Fix README to show graphics on PyPI (#2961) * Fix for PyPI Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Few more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update README.md * Update README.md --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add some Vale styles (#2963) * add some styles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Tweaks to styles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * more tweaks Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Revise some styles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Revise severity of some styles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * Update .github/styles/Kedro/words.yml Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Minor changes to test + release notes + docs Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update anyconfig requirement from ~=0.10.0 to >=0.10,<0.14 (#2876) Updates the requirements on [anyconfig](https://github.com/ssato/python-anyconfig) to permit the latest version. - [Changelog](https://github.com/ssato/python-anyconfig/blob/next/NEWS) - [Commits](https://github.com/ssato/python-anyconfig/compare/RELEASE_0.10.0...RELEASE_0.13.0) Signed-off-by: Jason Hite <jasonmhite@gmail.com> --- updated-dependencies: - dependency-name: anyconfig dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Move default template to static `pyproject.toml`, take 2 (#2853) * Remove leftover reference to micropkg setup.py Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Replace outdated extras requirement with kedro-datasets Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Remove outdated mention to environment.yml Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Move flake8 configuration to dedicated file Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Replace setup.py with pyproject.toml in default project template Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Adapt link to moving file in docs Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Revert "Replace outdated extras requirement with kedro-datasets" This reverts commit dfa8a9e8377ca60dcc3026e93087c5538791a0b5. Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> * Add release notes for pyproject.toml change Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> --------- Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add deprecation warnings to top-level use of layer in catalog definition (#2964) * add deprecation Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * lint Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Use FutureWarning Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * changes based on review Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * add link to docs Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> * Update RELEASE.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update on credentials.md (#2787) * Update on credentials.md Updating example code in credentials.md to make it usable when copy/pasting. Since as of now it will generate a TypeError Signed-off-by: Jose <jmnunezd123@gmail.com> * adding a more explicit code example, now we directly show the user that project_path is a pathlib.Path object Signed-off-by: Jose <jmnunezd123@gmail.com> * Update docs/source/configuration/credentials.md Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jose <jmnunezd123@gmail.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Nok Lam Chan <nok.lam.chan@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Cap pluggy 1.3 release (#2981) Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Make vale linter only run when PR opened or reopened (#2982) Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update merge-gatekeeper.yml (#2960) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Configure starters to use OmegaConfigLoader (#2974) * Change starters to use OmegaConfigLoader Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix linter error Signed-off-by: lrcouto <laurarccouto@gmail.com> * Suppress import outside toplevel linting for starters template Signed-off-by: L. R. Couto <laurarccouto@gmail.com> * Fix linter error Signed-off-by: lrcouto <laurarccouto@gmail.com> * Add changes to release notes Signed-off-by: lrcouto <laurarccouto@gmail.com> --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: L. R. Couto <laurarccouto@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Stop OmegaConfigLoader from reading config from hidden directory like `ipynb_checkpoints` (#2977) * Check plugins implement valid hooks Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Add release note Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Staging work - add custom functions to check hidden folder and files. Tests still failing Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix test - checkpoints should use the same environment Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Revert "Check plugins implement valid hooks" This reverts commit f10bede0401b80e3385e93458995835a2db721c9. * Update RELEASE.md Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com> * fix lint Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Ankita Katiyar <110245118+ankatiyar@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add migration steps for `TemplatedConfigLoader` to `OmegaConfigLoader` (#2904) Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Introduce a sentinel value _NO_VALUE to improve Global resolvers to support defaults `0` or `None` (#2976) * Add None support to globals Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add warning when default value is used Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Check keys Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Nok's suggestions Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Create the test to check the non-existing keys Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * add more tests to catch case when global key is not a dict Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Fix the null test Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Introduce sentinel value _NO_VALUE Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * rename test Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Improve error mesasge and raise InterpolationResolutionError when key does not exist and no default Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Fix non exist default test Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Fix test Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Use omegaconf to replace the custom resolving logic Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * uncommented some tests Signed-off-by: Nok <nok.lam.chan@quantumblack.com> * Remove dead code Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update error message Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Release 0.18.13 (#2988) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Fix docstrings on kedro/extras/datasets (#2995) * Fix docstrings on kedro/extras/datasets Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix formatting error Signed-off-by: lrcouto <laurarccouto@gmail.com> * Explicitly define code block Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix empty line under code block Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix formatting error Signed-off-by: lrcouto <laurarccouto@gmail.com> * Fix broken link Signed-off-by: lrcouto <laurarccouto@gmail.com> * Bump kedro-datasets version Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update links in partitioned and incremental datasets Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update links in partitioned and incremental datasets Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update links in partitioned dataset Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update links in partitioned dataset Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add polars.GenericDataSet to .rst Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Co-authored-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Minor docs changes on data section to create a PR and test Vale styles (#2966) * Minor changes to create a PR and test Vale styles Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> * fix some vale warnings Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> --------- Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * PiP/pyproject.toml and Conda/Meta.yaml Sync (#2922) * Resynced Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> * restore lower bound for pluggy Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> --------- Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * fix typo for build (#3001) Signed-off-by: rxm7706 <95496360+rxm7706@users.noreply.github.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Add hook example to access `metadata` (#2998) * Check plugins implement valid hooks Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * add Metadtahook Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> * Fix docs according to comments Signed-off-by: Nok <nok.lam.chan@quantumblack.com> --------- Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Bump release notes version Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Expand docs for customized pipeline templates. Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update release notes and docs Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update language linter to also run when PR converted ready for review Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Fix ci - Use `--resolver=backtracking` with `kedro build-reqs` in e2e tests (#3010) * Bump build version Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Remove telemetry from test default starter Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add package_name back Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Pin build only for 3.7 Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Try upgrade pip Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Add constraint Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Update for windows Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Run build-reqs with backtracking resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> * Run build-reqs with backtracking resolver Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update docs Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Update style to catch some more US spellings Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Make dataset factory resolve nested dict properly (#2993) Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Apply suggestions from code review Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Resolve suggestions from code review Signed-off-by: Jason Hite <jasonmhite@gmail.com> * Revert changes ukspelling.yml Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> --------- Signed-off-by: Jason Hite <jasonmhite@gmail.com> Signed-off-by: SajidAlamQB <90610031+SajidAlamQB@users.noreply.github.com> Signed-off-by: Ankita Katiyar <ankitakatiyar2401@gmail.com> Signed-off-by: Dmitry Sorokin <dmd40in@gmail.com> Signed-off-by: Merel Theisen <merel.theisen@quantumblack.com> Signed-off-by: laizaparizotto <laizamsparizotto@gmail.com> Signed-off-by: Jo Stichbury <jo_stichbury@mckinsey.com> Signed-off-by: Nok <nok.lam.chan@quantumblack.com> Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com> Signed-off-by: lrcouto <laurarccouto@gmail.com> Signed-off-by: Deepyaman Datta <deepyaman.datta@utexas.edu> Signed-off-by: Ahdra Merali <ahdra.merali…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Resolves #2849
Development notes
Checklist
RELEASE.mdfile