Update documentation of hooks LIFO order

[noklam]

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

fix #1940, the scope of this ticket is limited to the LIFO order, #1718 will be the ticket to summarise the order of kedro run

(Opening this PR to draft my idea and I will add the docs accordingly and update the title)
Few things I plan to do:

• We need to also document what the order is in which auto-discovered hooks run. Add this to: https://kedro.readthedocs.io/en/stable/hooks/introduction.html To verify the run order, create a project and install several plugins with hooks to test.
• Add a comment in the settings.py template file to explain the run order of hooks
• Check if https://github.com/pytest-dev/pytest/blob/196f01965ed264e1fd917db236c872f85403753d/doc/en/how-to/writing_plugins.rst#plugin-discovery-order-at-tool-startup helps or link to pytest for more detail
• Maybe this comment Document the kedro run lifecycle and hooks execution order #1718 (comment)
• https://docs.kedro.org/en/stable/extend_kedro/plugins.html#hooks - I think we missed to update the documentation when we move from pyproject.toml to setup.py. I want to keep both in the documentation since people aren't necessary moving to pyproject.toml (existing projects). I found this https://setuptools.pypa.io/en/latest/userguide/entry_point.html#entry-points-for-plugins
• Do the same for starters (After this Pr is approved) - Update LIFO hooks and pyproject.toml for hooks in starters kedro-starters#146
• Maybe update _register_hooks_setuptools, it's outdated.

Development notes

Checklist

• Read the contributing guidelines
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes
• Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

[noklam]

https://docs.kedro.org/en/stable/extend_kedro/plugins.html#community-developed-plugins And while I am looking at this, we have awesome-kedro now, maybe it make sense to shuffle the order accordingly or at least move the more stable one to the top? (kedro-mlflow and GetInDatas)

cc @stichbury

[stichbury]

https://docs.kedro.org/en/stable/extend_kedro/plugins.html#community-developed-plugins And while I am looking at this, we have awesome-kedro now, maybe it make sense to shuffle the order accordingly or at least move the more stable one to the top? (kedro-mlflow and GetInDatas)

cc @stichbury

That sounds OK to me, although it's hard to rank them by stability once we get past the most obvious. We did have a discussion a while ago that @idanov initiated about using a badge on individual plugin README pages to indicate when they were last updated. But anyway, let's remove the list from the docs and point instead to awesome-kedro where we list the plugins with GetInData starting the list and a rough order once we've included the most obviously stable.\

EDIT: See comment to the parent task, which should really be a response here

[noklam]

After some testing, I found out the LIFO order is quite tricky, and I don't recommend we document it as is.

The "registration order" is a combination of these factors:

1. Project hooks
2. hooks from entrypoints (which follow alphabetical order, and is a 2-level for loop

The execution order is the reverse of registration order (LIFO), except that user may use hook_impl "try_first" argument. Optionally, you can also import a registered entrypoints into project to change the order (i.e. importing kedro_viz hook in settings.py will force it to be registered earlier)

In general, order shouldn't be matter (99% of the time). If user need to control this, we should recommend using pluggy built-in functionality "try_first" argument. We shouldn't recommend to control the order via entrypoints order

[stichbury]

Suggested change

	Starting from 0.18.14, Kedro switch to replace `setup.py` with `pyproject.toml`. The plugin need to provide the entry points in either file.
	From version 0.18.14, Kedro replaced `setup.py` with `pyproject.toml`. The plugin needs to provide entry points in either file.

[stichbury]

I'm confused by this -- can you add the entry point in either file? Should you, or shouldn't we make a recommendation not to use setup.py since we've replaced it with pyproject.toml?

[noklam]

@stichbury We have moved to pyproject.toml if you created starter with the new version, but there are projects created with older version of Kedro and they only upgrade kedro but not the project template. Maybe its's too confusing and I should just keep the new one?

[stichbury]

Few changes suggested and I'll make a commit direct onto hooks.md on your branch if that's OK as I want to make a few changes there.

[stichbury]

@ankatiyar I know I've asked this before but @noklam has touched a few pages of markdown and there were some issues with his text but Vale didn't flag anything. I'm going to deliberately introduce an issue into hooks.md to see if it shouts...I'm not convinced it's running as it should be.

[merelcht]

Thanks for also updating the docs about plugins!

[merelcht]

I'm guessing you only added this for debugging? 😄

[noklam]

Thanks! 😅 I was working between different branches must have left this accidentally.

[stichbury]

LGTM, just a few changes which are mostly capitalisation of Hook 😆

[astrojuanlu]

oops, typo

[noklam]

I did test it with the plugin hooks🥲 maybe I mess it up when I copy-paste it back to the doc. Thanks for spotting this.

[astrojuanlu]

Shouldn't this be plugin:starters? (Same typo as above)