Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure Docs #27235

Merged
merged 10 commits into from
Dec 16, 2022
Merged

Restructure Docs #27235

merged 10 commits into from
Dec 16, 2022

Conversation

TohnJhomas
Copy link
Contributor

This PR is designed to restructure the overcrowded and confusing sidebar of the Astronomer Documentation. This is intended to allow for further scalability of the airflow docs, and to increase user-friendliness to new users.

The current docs have 28 top-level pages and two sub-sections of substance: "How-to Guides" and "Concepts." Here I've attempted to clean up the sidebar and make the structure more clear by both splitting "Concepts" into three new top-level pages ("Core Concepts," "Authoring and Scheduling" and "Administration and Deployment") and folding many of the current top-level pages into these categories.

The structure of the docs as they currently are is visually here: The left column is representative of the top-level sidebar, with the concepts subsection broken out to the right. How-to guides is another large section, but because it didn't undergo any substantial change I've hidden the sub-pages for ease of understanding.

Screen Shot 2022-10-24 at 1 12 51 PM

The structure of the refactored docs is below, with the same format as above:

Screen Shot 2022-10-24 at 1 12 41 PM


@boring-cyborg boring-cyborg bot added area:core-operators Operators, Sensors and hooks within Core Airflow provider:cncf-kubernetes Kubernetes provider related issues area:providers kind:documentation labels Oct 24, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented Oct 24, 2022

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst)
Here are some useful points:

  • Pay attention to the quality of your code (flake8, mypy and type annotations). Our pre-commits will help you with that.
  • In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
  • Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
  • Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
  • Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
  • Be sure to read the Airflow Coding style.
    Apache Airflow is a community-driven project and together we are making it better 🚀.
    In case of doubts contact the developers at:
    Mailing List: dev@airflow.apache.org
    Slack: https://s.apache.org/airflow-slack

@TohnJhomas
Copy link
Contributor Author

This version has known issues with cross-provider label referencing

@TohnJhomas TohnJhomas marked this pull request as draft October 24, 2022 19:17
@uranusjr
Copy link
Member

Thanks for taking the time to do this! The new structure makes much more sense to me.

@dstandish
Copy link
Contributor

Thanks for taking the time to do this! The new structure makes much more sense to me.

I second this emotion. Thanks

@potiuk
Copy link
Member

potiuk commented Oct 26, 2022

Conflicts to resolve I am avraid :( . Emotions are the same :)

@potiuk
Copy link
Member

potiuk commented Oct 26, 2022

Would be great also if you generate the docs breeze build-docs and share some screenshots.

@BasPH
Copy link
Contributor

BasPH commented Oct 26, 2022

Looking forward to this much-needed change! 👍

@potiuk
Copy link
Member

potiuk commented Oct 31, 2022

Looking forward to this much-needed change! 👍

Likewise.

@mik-laj
Copy link
Member

mik-laj commented Nov 3, 2022

Can you also update the redirects.txt file? It is essential for Google to be able to find the new page locations and also for users to be properly redirected when they change Apache Airflow versions in the documentation.

@TohnJhomas
Copy link
Contributor Author

Yeah - just pointing all the old locations at their new locations?

@mik-laj
Copy link
Member

mik-laj commented Nov 4, 2022

@TohnJhomas Yes. Exactly.

@dstandish
Copy link
Contributor

has conflicts

@BasPH
Copy link
Contributor

BasPH commented Dec 14, 2022

@TohnJhomas I'd love to see this PR merged, any chance you could resolve the conflicts?

@potiuk
Copy link
Member

potiuk commented Dec 14, 2022

@TohnJhomas I'd love to see this PR merged, any chance you could resolve the conflicts?

Oh yeah. I would very, very much love that too. Maybe you need some help @TohnJhomas ? I am happy to spend some time on helping to solve issue with docs building (I know it is a very daunting thing to fight with Sphinx. The way the tags etc. are defined are rather non-intuitive and there are many, many traps you can fall in.

@TohnJhomas
Copy link
Contributor Author

@potiuk / @BasPH Honestly - I got a little stuck working on some of the sphynx roles. I'm getting redirects.txt and the merge conflicts fixed today and everything pushed up to date. a few broken links to/from KubernetesPodOperator probably shouldn't hold everything else up

@potiuk
Copy link
Member

potiuk commented Dec 16, 2022

Approved workflow again :)

@potiuk
Copy link
Member

potiuk commented Dec 16, 2022

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

Let me merge it quickly before there are any conflicts

🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉 🎉

@potiuk potiuk merged commit 401fc57 into apache:main Dec 16, 2022
@boring-cyborg
Copy link

boring-cyborg bot commented Dec 16, 2022

Awesome work, congrats on your first merged pull request!

gschuurman pushed a commit to gschuurman/airflow that referenced this pull request Dec 19, 2022
A big restructuring of our docs to make them easier to reason about - in a much more logical and consistent way. Likel other updates and content changes will be needed to make it even better.
@pierrejeambrun pierrejeambrun added this to the Airflow 2.5.1 milestone Jan 9, 2023
@pierrejeambrun pierrejeambrun added the type:doc-only Changelog: Doc Only label Jan 9, 2023
ephraimbuddy pushed a commit that referenced this pull request Jan 12, 2023
A big restructuring of our docs to make them easier to reason about - in a much more logical and consistent way. Likel other updates and content changes will be needed to make it even better.

(cherry picked from commit 401fc57)
ephraimbuddy added a commit that referenced this pull request Jan 12, 2023
raboof added a commit to raboof/airflow that referenced this pull request Apr 24, 2023
Add a broad-strokes description of the security expectations
operator should expect. This will get included into
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/index.html

I'm no Airflow expert so help definitely welcome.

I would like this section to be a bit more prominent rather than hidden away
under 'Administration and Deployment', but it looks like the structure was
carefully considered in apache#27235 so this
is probably fine.

Eventually we could point the 'Security' link on the main pages
like https://airflow.apache.org/ to this page.
ashb pushed a commit that referenced this pull request Apr 25, 2023
Add a broad-strokes description of the security expectations
operator should expect. This will get included into
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/index.html

I'm no Airflow expert so help definitely welcome.

I would like this section to be a bit more prominent rather than hidden away
under 'Administration and Deployment', but it looks like the structure was
carefully considered in #27235 so this
is probably fine.

Eventually we could point the 'Security' link on the main pages
like https://airflow.apache.org/ to this page.
eladkal pushed a commit that referenced this pull request Jun 8, 2023
Add a broad-strokes description of the security expectations
operator should expect. This will get included into
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/index.html

I'm no Airflow expert so help definitely welcome.

I would like this section to be a bit more prominent rather than hidden away
under 'Administration and Deployment', but it looks like the structure was
carefully considered in #27235 so this
is probably fine.

Eventually we could point the 'Security' link on the main pages
like https://airflow.apache.org/ to this page.

(cherry picked from commit f20c08a)
ahidalgob pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Jun 12, 2023
Add a broad-strokes description of the security expectations
operator should expect. This will get included into
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/index.html

I'm no Airflow expert so help definitely welcome.

I would like this section to be a bit more prominent rather than hidden away
under 'Administration and Deployment', but it looks like the structure was
carefully considered in apache/airflow#27235 so this
is probably fine.

Eventually we could point the 'Security' link on the main pages
like https://airflow.apache.org/ to this page.

GitOrigin-RevId: f20c08a77c9daefe54c14c5013fcf53d84b5744d
ahidalgob pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Nov 7, 2023
Add a broad-strokes description of the security expectations
operator should expect. This will get included into
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/index.html

I'm no Airflow expert so help definitely welcome.

I would like this section to be a bit more prominent rather than hidden away
under 'Administration and Deployment', but it looks like the structure was
carefully considered in apache/airflow#27235 so this
is probably fine.

Eventually we could point the 'Security' link on the main pages
like https://airflow.apache.org/ to this page.

(cherry picked from commit f20c08a77c9daefe54c14c5013fcf53d84b5744d)

GitOrigin-RevId: bb86b468ae9396ece41f105db6104ee9282b4ed1
ahidalgob pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request May 15, 2024
Add a broad-strokes description of the security expectations
operator should expect. This will get included into
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/security/index.html

I'm no Airflow expert so help definitely welcome.

I would like this section to be a bit more prominent rather than hidden away
under 'Administration and Deployment', but it looks like the structure was
carefully considered in apache/airflow#27235 so this
is probably fine.

Eventually we could point the 'Security' link on the main pages
like https://airflow.apache.org/ to this page.

GitOrigin-RevId: f20c08a77c9daefe54c14c5013fcf53d84b5744d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:core-operators Operators, Sensors and hooks within Core Airflow area:providers kind:documentation provider:cncf-kubernetes Kubernetes provider related issues type:doc-only Changelog: Doc Only
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

9 participants