-
Notifications
You must be signed in to change notification settings - Fork 899
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update documentation of hooks LIFO order #3013
Conversation
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
https://docs.kedro.org/en/stable/extend_kedro/plugins.html#community-developed-plugins And while I am looking at this, we have cc @stichbury |
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
That sounds OK to me, although it's hard to rank them by stability once we get past the most obvious. We did have a discussion a while ago that @idanov initiated about using a badge on individual plugin README pages to indicate when they were last updated. But anyway, let's remove the list from the docs and point instead to EDIT: See comment to the parent task, which should really be a response here |
After some testing, I found out the LIFO order is quite tricky, and I don't recommend we document it as is. The "registration order" is a combination of these factors:
The execution order is the reverse of registration order (LIFO), except that user may use In general, order shouldn't be matter (99% of the time). If user need to control this, we should recommend using |
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
docs/source/extend_kedro/plugins.md
Outdated
@@ -29,8 +29,15 @@ def to_json(metadata): | |||
pipeline = pipelines["__default__"] | |||
print(pipeline.to_json()) | |||
``` | |||
Starting from 0.18.14, Kedro switch to replace `setup.py` with `pyproject.toml`. The plugin need to provide the entry points in either file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Starting from 0.18.14, Kedro switch to replace `setup.py` with `pyproject.toml`. The plugin need to provide the entry points in either file. | |
From version 0.18.14, Kedro replaced `setup.py` with `pyproject.toml`. The plugin needs to provide entry points in either file. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm confused by this -- can you add the entry point in either file? Should you, or shouldn't we make a recommendation not to use setup.py since we've replaced it with pyproject.toml?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@stichbury We have moved to pyproject.toml
if you created starter with the new version, but there are projects created with older version of Kedro and they only upgrade kedro
but not the project template. Maybe its's too confusing and I should just keep the new one?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few changes suggested and I'll make a commit direct onto hooks.md
on your branch if that's OK as I want to make a few changes there.
@ankatiyar I know I've asked this before but @noklam has touched a few pages of markdown and there were some issues with his text but Vale didn't flag anything. I'm going to deliberately introduce an issue into |
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for also updating the docs about plugins!
kedro/framework/hooks/manager.py
Outdated
@@ -73,6 +73,7 @@ def _register_hooks_setuptools( | |||
plugin_names = set() | |||
disabled_plugin_names = set() | |||
for plugin, dist in plugininfo: | |||
print("DEBUG!!!", plugin, dist) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm guessing you only added this for debugging? 😄
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! 😅 I was working between different branches must have left this accidentally.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a few changes which are mostly capitalisation of Hook 😆
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Signed-off-by: Nok Chan <nok.lam.chan@quantumblack.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
Co-authored-by: Jo Stichbury <jo_stichbury@mckinsey.com>
To add the entry point to `pyproject.toml`, the plugin needs to provide the following `entry_points` configuration: | ||
```toml | ||
[project.entry-points."kedro.project_commands"] | ||
kedrojson = kedrojson.plugin.commands |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oops, typo
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did test it with the plugin hooks🥲 maybe I mess it up when I copy-paste it back to the doc. Thanks for spotting this.
) | ||
```toml | ||
[project.entry-points."kedro.starters"] | ||
starter = plugin.starters |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't this be plugin:starters
? (Same typo as above)
Description
fix #1940, the scope of this ticket is limited to the LIFO order, #1718 will be the ticket to summarise the order of
kedro run
(Opening this PR to draft my idea and I will add the docs accordingly and update the title)
Few things I plan to do:
kedro run
lifecycle and hooks execution order #1718 (comment)pyproject.toml
tosetup.py
. I want to keep both in the documentation since people aren't necessary moving topyproject.toml
(existing projects). I found this https://setuptools.pypa.io/en/latest/userguide/entry_point.html#entry-points-for-plugins_register_hooks_setuptools
, it's outdated.Development notes
Checklist
RELEASE.md
file