ci: compress everything and deflate it on the server

[alandefreitas]

Uploading the website is now taking forever…

There’s more than the website on this server. The LLVM releases are also there. Still, storage and processing must be unrelated because the CPU is not busy, and because the LLVM release uploads (which are much larger) do not take this long.

This is likely because the website consists of numerous small pages. This tends to confuse servers. We should compress everything and then deflate it on the server.

[mizvekov]

This is sporadic. I just did a test push on a branch, measuring the time taken to upload everything: https://github.com/cppalliance/mrdocs/actions/runs/18201885493/job/51824406241

real	1m17.886s
user	0m0.496s
sys	0m1.385s

real	0m0.642s
user	0m0.013s
sys	0m0.006s

real	7m26.738s
user	0m2.079s
sys	0m7.430s

That's only about 8 minutes, a far cry from the almost two hours in https://github.com/cppalliance/mrdocs/actions/runs/18111852278/job/51541092312

We can certainly try to improve on those 8 minutes, but I suspect it won't have a significant effect on whatever causes it to go on for hours sporadically.

[alandefreitas]

So the code in a PR can update the website? Yes. We have to fix it. Please open an issue for that.

That's only about 8 minutes

Do you mean it was 8 minutes without doing anything or 8 minutes after doing what?

If you did nothing, what do you think was causing the 40 minutes? This has been happening for quite a while. Is there something in the workflow including more files when it's released?

[mizvekov]

No this is not running on a PR, in order to test this, I just pushed to a new branch on the main repo.

It was 8m47s total on the Publish website job, the numbers I quote above are the durations for the three ssh invocations we perform.

As far as I know, this workflow run includes everything a push to the develop branch would, the difference in time seems circumstantial rather than a drastic change in the amount of things pushed.

You can see on the numbers above, the far larger share is uploading the demos, but that's still only 7m27s.

[alandefreitas]

2 hours is crazy. I was thinking more like 40 minutes.

I have a suspicion this has been happening more often than usual. Honestly, I’m not even sure I really understood why you needed to measure the time like that since GitHub itself already shows you how long each step took, and that step is dominated by a single operation.

Either way, we’d need to go through https://github.com/cppalliance/mrdocs/actions?query=branch%3Adevelop to see what’s going on and figure out how circumstantial the problem really is. And of course, we can keep monitoring what happens whenever we update develop.

If it’s not that circumstantial, another possibility is that the develop version, for various reasons, is including more files in the package. The folder might not be the same because Antora has some considerations related to the current branch and we have some steps in the workflow that are conditioned on the branch.

In either case, we probably need two more issues.

• If that step really runs on every branch, we need to fix that. Only the branches listed for Antora (develop, master, v*..) need that step.
• And if we do monitor it and confirm it’s circumstantial, another issue would be to write down everything we already investigated, how circumstantial we found it to be, and then poke Sam to see if there’s anything else he can do or try.

[mizvekov]

I have a suspicion this has been happening more often than usual. Honestly, I’m not even sure I really understood why you needed to measure the time like that since GitHub itself already shows you how long each step took, and that step is dominated by a single operation.

There are several operations, and when there is a problem you never know for sure, so it's a safe bet to measure everything before changing it.

Either way, we’d need to go through https://github.com/cppalliance/mrdocs/actions?query=branch%3Adevelop to see what’s going on and figure out how circumstantial the problem really is. And of course, we can keep monitoring what happens whenever we update develop.

If it’s not that circumstantial, another possibility is that the develop version, for various reasons, is including more files in the package. The folder might not be the same because Antora has some considerations related to the current branch and we have some steps in the workflow that are conditioned on the branch.

In either case, we probably need two more issues.

• If that step really runs on every branch, we need to fix that. Only the branches listed for Antora (develop, master, v*..) need that step.

Well its not running on every branch by default, I added my branch in to the white list so I could test it.
There is no problem here.

• And if we do monitor it and confirm it’s circumstantial, another issue would be to write down everything we already investigated, how circumstantial we found it to be, and then poke Sam to see if there’s anything else he can do or try.

[alandefreitas]

There is no problem here.

Oh... Great. So only the second issue then. Please check how circumstantial this is and open another issue so Sam can either help us or we can at least know what we have already investigated.