Request for consistent access to kedro objects with plugins accross kedro versions

[WaylonWalker]

Description

Stability for pipeline devlopers

First off hats off to the Kedro team in making essentially no breaking changes over the last few years from the perspective of the pipeline developer. Ever since fsspec support landed the DataSets, Catalog Pipeline, Node, and Runner objects have been quite stable and able to simply bring along pipeline work to new versions with pretty low friction.

Many changes for plugin developers

From the perspective of the plugin developer, there is quite a dance to figure out how to get Kedro objects loaded to manipulate or observe. looking at kedro-viz there is quite a depth of knowledge one needs to have across all versions of Kedro to accurately deploy a plugin that is compatible with multiple versions.

Resolution

Can we get a version agnostic way to access these things whether it's through Kedro itself or through a meta-package? It looks like much of the work is already being done by the team to support kedro-viz. It would be nice if the community could leverage that work as well. I would be open to other solutions that you may have in mind.

[merelcht]

Hi @WaylonWalker sorry about the late reply. We've had a chat about this issue with the team. In the short term, we will improve the documentation for plugin developers and look at adding more complex examples that showcase how this workflow can be achieved. In the long term we'd need to do user research with the plugin developers to find what the best resolution would be for this. However, since this is not a high priority for the team for the time being, I'll close this issue in favour of the short term solution we'll be able to tackle in a more reasonable time frame: #1539.

I hope that's okay, and as always thank you so much for working on improving Kedro! ❤️

[WaylonWalker]

A documented way to consistently access kedro objects would be much appreciated, thanks @MerelTheisenQB