Migrate to 0.16.5

[Galileo-Galilei]

The 0.16.5 release is just out and comes with good and bad news:

• Many tests are broken (for pipeline_ml, hook and template) and should be fixed ASAP. However, I think we can release all the functionalities we've worked on in the past month to make them available in a 0.3.0 release, and plan to migrate after this.
• The new way to register the config_loader, the pipeline and the catalog make them accessible in a global _hook_manager variable instantiated by get_hook_manager(). This enable us to change get_mlflow_config() which currently:
  • depends heavily on the template, and can only be called when the current working directory is the root of the kedro project. Indeed, when we call this function inside a hook we have no way to access the project path except by reading the .kedro.yml, but this requires knowing in advance where it is. Thanks to this change, we will be able to access the ConfigLoader instantiated in the ProjectContext inside the hook. This is directly related to this issue: [KED-2143] Adding a ConfigLoader instance into hook specs params kedro-org/kedro#506, and it will enable to fix get_mlflow_config use the working directory instead of given path when called within load_context #30 which is particularly annoying but could not be fixed before this change.
  • break supercharging of config loader by a user (which is quite common for people using TemplatedConfigLoader who are not able to use their customisation within the "mlflow.yml" file).
• Update the kedro mlflow init command according to this new template. Should it write in the run.py ProjectContext class or in the kedro.yml hooks key?

[Galileo-Galilei]

Does anyone wants to take this one? @kaemo @akruszewski @takikadiri? Only the first point though since the second point has bee splitted between #66 and #72 .

[turn1a]

@Galileo-Galilei I could work on this one next week.

[Galileo-Galilei]

Yes, sure. Do you plan to finish #56 first? I updated the milestones to give visibility to what we prioritize (accordingly to what discussed), but feel free to move it to next release if you're not good to go yet.

[turn1a]

@Galileo-Galilei I've got #56 mostly done so I'll finish it first.

[takikadiri]

Almost all kedro_mlflow features and their tests are now working/passing on kedro 0.16.5, except kedro mlflow init because of the run.py update part.