feat: add dags.gitSync.submodules value

[karakanb]

Signed-off-by: Burak Karakan burak.karakan@gmail.com

What issues does your PR fix?

• N/A

What does your PR do?

This PR introduces a new value dags.gitSync.submodules (default: recursive) to control the submodule strategy for git-sync pods.

In my use case, the repo that is being cloned has a lot of submodules, but I would rather not init the submodules because it takes 20+ seconds, therefore I want to be able to disable the submodules to significantly speed up my tasks.

Checklist

For all Pull Requests

• Commits are signed off
• Commits have semantic messages
• Documentation updated
• Passes ct linting

For releasing ONLY

• Chart.yaml version bumped
• CHANGELOG.md updated

[karakanb]

@thesuperzapper should I update the version with this as well? would you consider this as a minor version or a patch version?

[stale]

This issue has been automatically marked as stale because it has not had activity in 60 days.
It will be closed in 7 days if no further activity occurs.

Thank you for your contributions.

---

Issues never become stale if any of the following is true:

1. they are added to a Project
2. they are added to a Milestone
3. they have the lifecycle/frozen label

[karakanb]

@thesuperzapper do you have any plans to merge / release this?

[thesuperzapper]

@karakanb thanks for the PR, don't worry, I haven't forgotten it!

As it adds value, it will need to be part of 8.7.0, my main goal for 8.7.0 is to have the task-aware autoscaler (#339), which is taking an incredible amount of work to get done but is progressing!

If enough feature PRs build up before then, we might cut 8.7.0 early which would probably include this PR.

[karakanb]

@thesuperzapper hey Mathew, I hope you are doing well. Do you happen to have a release schedule in mind? it feels like smaller changes like these can go as part of patch versions as long as they are backwards compatible, and would avoid blocking other smaller features until the big ones you'd like to have are ready, as well as avoiding the need to fork by others.

Would you be interested in releasing some patch versions with fixes like this, as well as support for newer k8s versions for issues such as #679?

[thesuperzapper]

@thesuperzapper hey Mathew, I hope you are doing well. Do you happen to have a release schedule in mind? it feels like smaller changes like these can go as part of patch versions as long as they are backwards compatible, and would avoid blocking other smaller features until the big ones you'd like to have are ready, as well as avoiding the need to fork by others.

Would you be interested in releasing some patch versions with fixes like this, as well as support for newer k8s versions for issues such as #679?

@karakanb sorry about the delay in getting this PR in a release, I have been busy moving countries!

So I can prioritize what to include, what are the critical fixes/features which you would expect in the next release?

[karakanb]

hey, no worries at all. Congrats on the country change, I hope all goes well for you! 🤞

for the next release, the only critical issue I have at the moment is this one: #679

afterwards, if a smaller release brought these that'd be nice

• nice to have: this PR
• nice to have: separate triggerer deployment
• nice to have: separate dag processor deployment: feat: add dag processor deployment #577

I believe the smaller releases with easy-to-upgrade versions would allow quick iteration + safe rollouts as every change would be gradually adopted by the community instead of bigger / riskier releases in that sense.

Thanks a lot for your efforts!

[thesuperzapper]

@karakanb thanks for the PR, there are only a few minor changes (see comments) that I think we need.

Also, we should update the "Load Airflow DAGs" FAQ page to show the new submodules value in its examples.

[karakanb]

@thesuperzapper this one is ready as well, please give it a look.

[thesuperzapper]

@karakanb thanks for the PR, I will be releasing this today in version 8.7.0 of the chart.