Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An initial rework of the "Concepts" documentation #15444

Merged
merged 1 commit into from Apr 29, 2021

Conversation

andrewgodwin
Copy link
Contributor

This is the first part of me trying to improve Airflow's documentation to make it more consistent and understandable.

This takes the old, single-page Concepts document and turns it into a new Concepts section, rewrites some of the text to have consistent terminology (upstream/downstream, Task vs Operator, etc.) and tries to make it generally more accessible and high-level. It also moves a couple of other pages in that directly fit with the existing content (Scheduler and Smart Sensors).

Once this is in, I would then like to take on a high-level reorganisation of the other documents, grouping them a bit better (installation/deployment, tutorials, howto) and also making a better overall concept split between "reference" documentation (exhausive, technical details and important notes - not autogenerated API docs) and "concept" documentation (high-level design explanation).

A considerable amount of what is currently in "Concepts" should actually be in a "Reference" section, in my view, but I've kept the current grouping for now to make this PR even slightly approachable.

@boring-cyborg boring-cyborg bot added area:core-operators Operators, Sensors and hooks within Core Airflow area:Scheduler Scheduler or dag parsing Issues kind:documentation labels Apr 19, 2021
specific language governing permissions and limitations
under the License.

Architecture Overview
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure if this page is well split. We on this page now have mixed up information on high-level architecture, as well as code snippets. Can you check it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's only two code snippets in there, mostly to illustrate the dependency declarations - what else would you be looking for?

Copy link
Member

@uranusjr uranusjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not read the concepts documents, but here are some very minor suggestions

docs/spelling_wordlist.txt Outdated Show resolved Hide resolved
docs/spelling_wordlist.txt Outdated Show resolved Hide resolved
docs/apache-airflow/executor/index.rst Outdated Show resolved Hide resolved
@andrewgodwin andrewgodwin force-pushed the docs-concepts branch 3 times, most recently from 3ee194a to a5dfecd Compare April 26, 2021 23:09
@kaxil kaxil requested a review from mik-laj April 27, 2021 19:29
@github-actions
Copy link

The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not modify the core of Airflow. If the committers decide that the full tests matrix is needed, they will add the label 'full tests needed'. Then you should rebase to the latest master or amend the last commit of the PR, and push it with --force-with-lease.

@github-actions github-actions bot added the okay to merge It's ok to merge this PR as it does not require more tests label Apr 27, 2021
@ashb ashb merged commit df143ae into apache:master Apr 29, 2021
@ashb ashb deleted the docs-concepts branch April 29, 2021 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core-operators Operators, Sensors and hooks within Core Airflow area:Scheduler Scheduler or dag parsing Issues kind:documentation okay to merge It's ok to merge this PR as it does not require more tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants