CI: Building docs in GitHub actions

[datapythonista]

Building the documentation in GitHub actions.

In Azure-pipelines we publish the built docs to GitHub pages, served at dev.pandas.io.

After this is merged, I'll work on publishing the Actions docs to the new OVH server. Possibly, publishing the docs of every PR.

[jbrockmendel]

The npdev build should now be fixed

[jorisvandenbossche]

Cool!

The artifacts are now uploaded as an archive, which means you can download that from the "Checks" tabs and in this way unzip to check locally how the built docs look like. But do you know if it is also possible with Github Actions to actually put the html files somewhere online so we can preview the built web/docs of a PR online?

[datapythonista]

But do you know if it is also possible with Github Actions to actually put the html files somewhere online so we can preview the built web/docs of a PR online?

I'm not sure yet, but as I mentioned in the description I think it may be possible (and easy) to publish with our new hosting. GitHub is also studying if there is a way they can do it themselves with the artifact. In a follow up I'll need the new hosting key to be added to GitHub (only admins can do it), and I'll give it a try, it may be trivial with GitHub actions. If it doesn't work another option is to use a custom GitHub app, but that's more costly to implement.

[datapythonista]

Looks like publishing the docs of a PR is quite straight forward with GitHub actions and the new hosting. Already made the first successful test.

@pandas-dev/pandas-core is https://dev.pandas.io/pr/29874/ a good location to access the docs for a PR, or does anyone have a better idea?

[jorisvandenbossche]

Ah, sorry, didn't read the last sentence very well (was assuming it was to publish the web/docs of master on the OVH server).

But yet, if hosting it at dev.pandas.io is straightforward, let's do that. Your proposed url sounds good for me.

(the reason I was thinking that GitHub would be nicer, if they would offer the hosting of artifacts, it that then they can deal with the storage / care about deleting older artifacts at some point, etc.)

[datapythonista]

I spoke to GitHub about this, and they're checking what can be done to publish the artifacts directly. I already have an action implemented that will delete pr web/docs once a PR is merged and closed. And I think we can schedule a nightly one to catch and delete any that for whatever reason stays there after a certain period of time or after its PR is closed.

Merging this now, and moving forward with publishing for now in the new hosting, and if/when GitHub is able to publish artifacts via http, we can move to that.

[datapythonista]

Looks like I misunderstood how the permissions of GitHub secrets work, but they have the same limitations as Azure pipelines. So, we can't publish the PR docs the way I was trying.

For publishing the web/docs on merges to master, I guess what makes sense is to publish the web directly in production, and the docs in a directory https://pandas.io/docs/dev/ or similar.

If there is agreement I'll do that in a follow up. I prefer to see that this is working as expected first. It'll probably be a bit tricky to independently sync two trees one inside the other with rclone.

[WillAyd]

Should this replace the Web_and_Docs build in pipelines?

[datapythonista]

Eventually, but the plan is to run it in parallel for a while until we're confident with the GitHub Actions job and the new hosting.

[datapythonista]

Merging this soon, so we can move forward with the new hosting.

[jorisvandenbossche]

Thanks @datapythonista !
Question: this is actually already uploaded when it is running on master? (is it what can be seen at https://pandas.io/ ?)

[datapythonista]

Yes, that's correct.

It's obviously not final, since /docs/ is now dev, and I guess we want it to be the stable version (or have another path).

We've already got the pandas email address. I'll try to find time in the next days to move the OVH account to it, so you and everybody else can access it.