Doc: Add FAQ to speed up parsing with tons of dag files #17519
Conversation
|
Might be worth mentioning this only helps with "simple" DAGs, e.g. if the main DAG file itself actually changes. Not sure exactly how to phrase it in this context, but this won't always be a safe option. |
docs/apache-airflow/faq.rst
Outdated
| @@ -119,6 +119,18 @@ How do I trigger tasks based on another task's failure? | |||
|
|
|||
| You can achieve this with :ref:`concepts:trigger-rules`. | |||
|
|
|||
| When there are a lot of dags files, how to speed up parsing of new files? | |||
There was a problem hiding this comment.
Should we be more specific to what is a lot?
This feature was added in apache#16075. This PR adds it to docs to avoid situations like apache#17437 closes apache#17437
Updated |
github-actions
bot
added
the
okay to merge
|
The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide that full test matrix is needed and add the 'full tests needed' label. Then you should rebase it to the latest main or amend the last commit of the PR, and push it with --force-with-lease. |
This feature was added in #16075. This PR adds it to docs to avoid situations like #17437
closes #17437
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code change, Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in UPDATING.md.