CircleCI takes an inordinate amount of time to complete.

[kchawla-pi]

make html is one process and complex to parallelize across containers + if we do manage that across all 4 containers, no one will be able to test their push until one of the containers completes and becomes unoccupied.

One strategy is to rebuild only the changed html pages in CircleCI, something sphinx/sphinx-gallery does by default, this will cut down the build time from hours to minutes.

To make sure that we don't miss a problem due to this, CircleCI might get another job, run once a day or once every three days, where the entire documentation is rebuild from scratch.

I intend to work on this soon as Nilearn 0.5b releases.

[kchawla-pi]

Expanding on the idea,

Ensuring internal changes don't break something:
The CI run after push is to ensure that internal changes (the PRs) aren't breaking something, which is best done in the same environment. TOt hat end I will create a custom docker image with the same environment used every time after a push.

Ensuring external changes don't break something:
The second workflow will be CI run daily and on master, which will be rebuilt from scratch the way it currently is. This will ensure our documentation is not breaking due to changes URLs, corrupted datasets, downed servers etc.

[ymzayek]

I will close this issue as it seems to have been completed. Further optimization of CircleCI builds can be discussed in a new issue.