Add deprecation notice to archived starters

[ankatiyar]

NOTE: Kedro datasets are moving from kedro.extras.datasets to a separate kedro-datasets package in
kedro-plugins repository. Any changes to the dataset implementations
should be done by opening a pull request in that repository.

Description

Close #3114

Development notes

• Show deprecation notice if the selected starter is scheduled for archiving
• Show deprecation notice when kedro starter list is used
• Add a message about the upcoming add-ons to kedro new command.

Developer Certificate of Origin

We need all contributions to comply with the Developer Certificate of Origin (DCO). All commits must be signed off by including a Signed-off-by line in the commit message. See our wiki for guidance.

If your PR is blocked due to unsigned commits, then you must follow the instructions under "Rebase the branch" on the GitHub Checks page for your PR. This will retroactively add the sign-off to all unsigned commits and allow the DCO check to pass.

Checklist

• Read the contributing guidelines
• Signed off each commit with a Developer Certificate of Origin (DCO)
• Opened this PR as a 'Draft Pull Request' if it is work-in-progress
• Updated the documentation to reflect the code changes
• Added a description of this change in the RELEASE.md file
• Added tests to cover my changes
• Checked if this change will affect Kedro-Viz, and if so, communicated that with the Viz team

[SajidAlamQB]

LGTM, thank you @ankatiyar! 👍

[datajoely]

Sorry I missed this - what is the replacement for the pyspark starter? This affects most of the QB Verticals

[SajidAlamQB]

Sorry I missed this - what is the replacement for the pyspark starter? This affects most of the QB Verticals

I believe it will be replaced by, https://github.com/kedro-org/kedro-starters/tree/main/spaceflights-pyspark, it was introduced by @merelcht with a new suite of starters that use spaceflights as a base.

[merelcht]

It would also be useful to add the deprecation mention to kedro starter list so the result is something like:

astro-airflow-iris:
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: astro-airflow-iris
astro-iris:
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: astro-airflow-iris
databricks-iris:
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: databricks-iris
pandas-iris (deprecated):
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: pandas-iris
pyspark (deprecated):
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: pyspark
pyspark-iris (deprecated):
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: pyspark-iris
spaceflights:
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: spaceflights
standalone-datacatalog (deprecated):
  template_path: git+https://github.com/kedro-org/kedro-starters.git
  directory: standalone-datacatalog

[datajoely]

So I want to strongly argue we shouldn't delete the empty pyspark starter- the verticals need any empty template to build off

[datajoely]

This would be a breaking change for many QB Vertical workflows

[ankatiyar]

@datajoely I was under the impression that you can create an empty pyspark template with the add-ons workflow, I think @SajidAlamQB is working on it?

[datajoely]

This would break the existing Alloy workflow - can you follow all the add on steps from a single command? It won't work if it's an interactive pattern.

[ankatiyar]

This would break the existing Alloy workflow - can you follow all the add on steps from a single command? It won't work if it's an interactive pattern.

@datajoely Yeah, I believe it'll be kedro new --addons=pyspark or something like that but @SajidAlamQB @AhdraMeraliQB can comment more on it.

[datajoely]

Either way it would make things a lot easier to keep this existing cookie cutter version I think

[datajoely]

The main risk is that existing users refuse to go to 0.19.x because of this

[SajidAlamQB]

@datajoely I was under the impression that you can create an empty pyspark template with the add-ons workflow, I think @SajidAlamQB is working on it?

Yes, the new add-ons flow will have an option to create an empty pyspark template, and yes I believe @lrcouto is working on #2873 which will allow you to use add-ons from a single command.

[datajoely]

I would strongly request that we keep both the pyspark and blank Kedro starters in cookiecutter format as is

[stichbury]

LGTM 💯

[merelcht]

Discussed with the team that the deprecation warnings will still go in, because they're blocking the 0.18.14 release. I will connect with @datajoely offline to discuss the best approach of serving Alloy/verticals needs with the new project creation flow and starter structure.