Disable parallel build #14796
Merged
Disable parallel build #14796
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
raydouglass
approved these changes
Jan 19, 2024
bdice
approved these changes
Jan 19, 2024
|
/merge |
rapids-bot bot
pushed a commit
that referenced
this pull request
Jan 26, 2024
cudf docs are generally very slow to build. This problem was exacerbated by the recent addition of libcudf C++ API documentation to the Sphinx build. This PR aims to ameliorate this issue for both local and CI builds by making the following changes: - The XML parsing logic used to clean up doxygen XML now avoids rewriting files unless they are actually modified. This prevents Sphinx from doing extra work during a second (text) build after the first (HTML) build. - toctrees on the generated API pages are removed (see https://pydata-sphinx-theme.readthedocs.io/en/stable/user_guide/performance.html#selectively-remove-pages-from-your-sidebar). - Text docs are disabled in PRs and only occur in nightly/branch builds. The net result is roughly a halving of the CI run time for the builds (~40 min to ~20 min). Further potential optimizations: - Reenabling parallel builds. We cannot fully revert #14796 until the theme is fixed, but if we can put in a warning filter we could reenable parallelism and have it work on just the reading steps of the build and not the writes. That would still improve performance. - Better caching of notebooks. [nbsphinx supports caching](https://myst-nb.readthedocs.io/en/latest/computation/execute.html#execute-cache), but there are various caveats w.r.t. 1) local vs CI builds, 2) proper cache invalidation, e.g. when notebook source does not change but underlying libraries do, and 3) forcing rebuilds. Alternatively, we could enable some environment variable that allows devs to turn off notebook execution locally. Making it opt-in would make the default behavior safe while providing an escape hatch for power users who want the builds to be fast. Authors: - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Lawrence Mitchell (https://github.com/wence-) URL: #14856
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
A new release of
pydata_sphinx_themefrom last night includes pydata/pydata-sphinx-theme#1642, which marks the theme as unsafe for parallel writing.Checklist