Add list of prerequisites that new provider package should fulfill

[potiuk]

Description

We need to have a folder template (ideally) and list of prerequisites that each provider should fulfill when it is created.

Use case / motivation

To make it easy to add new provider and to make new contributors aware what is needed in order to add provider. The following should be added:

• directory structure (possibly automated templates or script to create them)
• usage scenarios for connections in hooks (see discussion in Standardising hooks / connections and adding TTLs #12128)
• Information about what is required:
  • hooks
  • operators/sensors/transfers
  • unit tests with mocks
  • integration tests if applicable
  • example dags for examples and their use (examples, howtos, system tests)
  • howto guides
• relation with backport providers.

[pcandoalmeida]

Hi @potiuk 🙂 I was thinking about taking a look at this; would you mind if I asked to help clarify the issue a bit further? Would this require a Gitub repository PR template update and documentation or something completely different? Apologies if I am missing the obvious here!

[potiuk]

It's rather about folder structure. We have monorepo - providers are in 'providers' packages/folders - both 'airflow' and 'tests' subfolders and they follow certain structure

[potiuk]

Ah sorry - did not understand. I think it is mostly a 'CONtRIBUTING.rst' update or rather creating new 'how to create a new provider.rst' file + folder structure explanation( or maybe a script creating a new provider with some scaffolding)

[pcandoalmeida]

Thanks for the clarification @potiuk. I can certainly try working on the documentation to begin with if that suits.

[TobKed]

Any updates for this issue? @pcandoalmeida are you planning to come back working on this? I am happy to help with it.
I opened PR with new operator (Apache Beam operators) and information about prerequisites for new providers will be very helpful.

[TobKed]

Is it not solved by #12921 ?

[pcandoalmeida]

Hey @TobKed sorry for the delay on this! @potiuk shall we close this issue?