New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
An initial rework of the "Concepts" documentation #15444
Conversation
specific language governing permissions and limitations | ||
under the License. | ||
|
||
Architecture Overview |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure if this page is well split. We on this page now have mixed up information on high-level architecture, as well as code snippets. Can you check it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There's only two code snippets in there, mostly to illustrate the dependency declarations - what else would you be looking for?
8f50c0a
to
40d4d50
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did not read the concepts documents, but here are some very minor suggestions
3ee194a
to
a5dfecd
Compare
a5dfecd
to
3e08e40
Compare
The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest master or amend the last commit of the PR, and push it with --force-with-lease. |
This is the first part of me trying to improve Airflow's documentation to make it more consistent and understandable.
This takes the old, single-page Concepts document and turns it into a new Concepts section, rewrites some of the text to have consistent terminology (upstream/downstream, Task vs Operator, etc.) and tries to make it generally more accessible and high-level. It also moves a couple of other pages in that directly fit with the existing content (Scheduler and Smart Sensors).
Once this is in, I would then like to take on a high-level reorganisation of the other documents, grouping them a bit better (installation/deployment, tutorials, howto) and also making a better overall concept split between "reference" documentation (exhausive, technical details and important notes - not autogenerated API docs) and "concept" documentation (high-level design explanation).
A considerable amount of what is currently in "Concepts" should actually be in a "Reference" section, in my view, but I've kept the current grouping for now to make this PR even slightly approachable.