[ENHANCEMENT] Distribute requirements.txt into folder wise structure

[ramitsurana]

Is your enhancement request related to a problem? Please describe.
No

Describe the enhancement you'd like
I was going through this File and wanted to check your thoughts if we could create a folder wise tree-like structure for different requirements like extensions, providers etc. in text files, depending on our use case and refer to the path of the requirements files from contants.py. Let me know if you see any challenges. For Example -

requirements
-> base.txt
-> providers
    -> gcp.txt
    -> azure.txt
    -> aws.txt
-> datasources
    -> huggingface.txt

How do you solve your current problem with the current status-quo of ZenML?
It would make it easier to update and maintain the versions of dependencies including extensions, base dependencies etc rather than updating the python file for any version updates.

Additional context

[bcdurak]

Hey @ramitsurana , thank you for opening this issue.

The integrations are an essential part of ZenML and it is definitely important to keep the requirements of each integration as organized as possible. However, with the increasing number of integrations, it will become more and more challenging to place them in clear-cut categories. Huggingface is a great example of this particular problem because we use it both as a datasource and as a training integration. That is why we opted to keep the requirements in a single central file.

[ramitsurana]

@bcdurak Cool. Makes Sense. Thanks for the explanation.