Python: Split Python docs

[Fokko]

This PR will split the Python docs into a separate site. The main reason for this is that the docs are part of the Java release, which is not in sync with the Python release cycle. Meaning that there is a high probability that the docs do not match with the current version of the code.

This will publish the docs to Github pages, by pushing this to the gh-pages branch. We can set up an alias from Apache, and point pyiceberg.apache.org to the GitHub pages endpoint.

I also tried readthedocs, but I found that not straightforward. Mostly because they have a build process on their end that will pull the code, and build the docs. This involves another pipeline that we have to monitor, and we have to set up webhooks. I am a simple man, and I like simple things, therefore I went for mkdocs. This can push the docs to GitHub pages in a single command: https://www.mkdocs.org/user-guide/deploying-your-docs/#project-pages

Considerations:

• Decided to keep it to a single page, for now, we can break it out into different pages later on. Let me know what you think of this.
• We build the docs now when we push to master, probably we'll change this later to trigger on tags.
• I've removed the Python docs from the other docs to avoid confusion and make sure that we have a single source of truth.

An example is shown here: https://fokko.github.io/incubator-iceberg/ (Once this is merged, I'll remove that one)

python/README.md will be shown on PyPi, and refers to the documentation.

[Fokko]

The deploy CI is failing, but I'm confident that we can make it work. Arrow has something similar. I'd love to know what other people think 👍🏻

[dhruv-pratap]

The deploy CI is failing, but I'm confident that we can make it work. Arrow has something similar. I'd love to know what other people think 👍🏻

I prefer this clean split, especially given the different release cycle for pyiceberg.

Just a minor nit-pick: the two levels of docs folder is a bit confusing here python/docs/docs/index.md
We should rename one of those folders to something more explicit. Maybe something like python/mkdocs/docs/index.md or python/docs/content/index.md

[Fokko]

Hey @dhruv-pratap thanks, I didn't think of that, but it makes a lot of sense. I've changed docs/docs to mkdocs/docs because the inner docs directory is part of the standard mkdocs structure.

[dhruv-pratap]

Looks good to me now.

[rdblue]

Why not use mkdocs gh-deploy? That should automatically push to gh-pages.

[Fokko]

I wasn't sure what mkdocs gh-deploy is doing under the hood, therefore I made the steps explicit.

[rdblue]

Do we want to publish for pull requests? Why not just master?

[Fokko]

That was mostly for testing, removed that one 👍🏻

[rdblue]

Can we add paths here with python/docs/**?

[Fokko]

Good call 👍🏻

[rdblue]

What's happening here?

[Fokko]

This is my personal favorite. This will actually detect shell, python, and config blocks in the code, and format it automatically. This way the code, script, and config will be nicely formatted 👍🏻 If it isn't formatted properly, it will fail the CI.

[rdblue]

Sounds reasonable. In the past, I've just configured those things in mkdocs, but if it's working that's great.

[rdblue]

Everything looks good to me. I have a few questions about cleaning up the github action, but that can be done async.