Fix oversubscription issue with lit precompile, label hack by alliepiper · Pull Request #6554 · NVIDIA/cccl

alliepiper · 2025-11-07T20:10:51Z

Moves the C2H tests earlier in the config so we can add the fake dependency to lit.
Explains and labels the hack we're using to avoid oversubscription.

Moves the C2H tests earlier in the config so we can add the fake dependency to lit. Explains and labels the hack we're using to avoid oversubscription.

copy-pr-bot · 2025-11-07T20:10:54Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

alliepiper · 2025-11-07T20:11:42Z

libcudacxx/test/libcudacxx/CMakeLists.txt


 if (MSVC)
-  # sccache cannot handle the -Fd option generationg pdb files
-  set(CMAKE_MSVC_DEBUG_INFORMATION_FORMAT Embedded)


Drive-by fix -- we define this at the CCCL level, no need to set it here. It doesn't have any effect on the lit tests defined in this file anyway.

miscco · 2025-11-10T07:45:09Z

libcudacxx/test/libcudacxx/CMakeLists.txt

+    # HACK: There is no way to tell CMake/ninja to always build a target serially,
+    # so we make this target depend on all other libcudacxx targets to avoid oversubscribing
+    # the build machine.
+    # FIXME: This has nasty side effects:
+    # - It's fragile and must be updated every time we add new targets to libcudacxx
+    # - It oversubs `-dev` presets that configure libcudacxx alongside other CCCL projects
+    # - It makes it impossible to just build this target alone since it brings in the world
+    # See related issue https://github.com/NVIDIA/cccl/issues/6163.
+    DEPENDS
+      libcudacxx.test.public_headers
+      libcudacxx.test.internal_headers
+      libcudacxx.test.public_headers_host_only
+      libcudacxx.test.c2h_all


Question: Why arent we just limiting the number of parallel compilations via the -j flag?

That would force the entire build to be serial. We can have parallelism in both ninja and lit, the issue is having them overlap.

The long-term solution for this issue is #6163. This PR just labels the hack that already exists.

github-actions · 2025-11-10T22:05:16Z

🥳 CI Workflow Results

🟩 Finished in 1h 16m: Pass: 100%/90 | Total: 15h 09m | Max: 1h 08m | Hits: 99%/219115

See results here.

No changes requested in review, was just a question.

github-actions · 2025-11-18T18:46:10Z

Only merged pull requests can be backported.

* Remove MSVC hint that is applied globally in CCCL. * Fix oversubscription issue with lit precompile, label hack. Moves the C2H tests earlier in the config so we can add the fake dependency to lit. Explains and labels the hack we're using to avoid oversubscription.

* Improvements to inspect_changes CI functionality. (#6535) 1. Rewrote `inspect_changes.sh` to python. 2. Split out project name, path, dependency information into new yaml file. 3. Simplified dependency specification (only direct dependencies are needed). 4. Split dependency specification into two types: full (use the pull_request matrix) and lite (use the pull_request_lite matrix). 5. Use new features to split some projects with expensive dependency chains into 'public' (public headers, etc) and `internal` (tests/examples/infra, etc). Dependencies are only added when the public API files change. 6. Update the ignored paths to include newer additions. 7. Add tests for inspect_changes to make it easier to test and validate modifications. * Add deps on thrust/cub to libcudacxx. (#6694) Complete the circle. * Fix oversubscription issue with lit precompile, label hack (#6554) * Remove MSVC hint that is applied globally in CCCL. * Fix oversubscription issue with lit precompile, label hack. Moves the C2H tests earlier in the config so we can add the fake dependency to lit. Explains and labels the hack we're using to avoid oversubscription. * Make missing sccache nonfatal. (#6582) * Add nvbench_helper tests to CI. (#6679) * Drive-by fix to packaging test script. Building isn't needed at the moment, but this will save some headaches if we add any executables that need to be built to this preset. * Fix warnings in nvbench_helper tests. * Skip test sizes that OOM CI runners. * Update boost dep to work with CMake 4+ * Enable CI coverage for nvbench_helper tests. * Update inspect_changes smoke tests. * Update libcudacxx C++ dialect handling. (#6693) * Switch preprocessor cache to S3. (#6561) This is more robust than the github cache approach, which evicts caches regularly due to a 10GB repo limit. The S3 cache will be more reliable and persistent. This also allows preprocessor caching in our linux devcontainers, improving developer experience with faster build times. * Restore libcudacxx dialect presets. (#6705) These were removed in #6693. * Remove special dialect handling from cudax build system. (#6702) * Remove special handling of C++ dialect in CUB's build system. (#6713) * Remove special handling for dialect in Thrust's build system. (#6722) * Remove special handling for dialect in Thrust's build system. * Allow consumers to set CCCL_TOP_LEVEL_PROJECT. This enables python to build the c libraries it depends on. * Force windows python builds to use ninja. * Exclude python build artifacts from git. * Fix python build to reuse CCCL's existing install rules. The old implementation reinvented the wheel and depended on variables that can't be relied upon. * Fix CUB header extensions. We use .cuh, not .h in CUB. This broke our install rules. * Fixup ptx_json header testing. * Fix issue with libcudacxx header tests. (#6785) Did some drive-by cleanups/standardizing of how our internal libraries link together. * Improve CMake package handling, add MSVC compat flags to libcudacxx's public interface. (#6791) * Change the MSVC preprocessor check to an error. When `/Zc:preprocessor` is not set, the build will error out before `#pragma message` directives are handled. Changing to an error ensures that this will be caught. * Add /Zc:preprocessor and /Zc:__cplusplus to public libcudacxx::libcudacxx target. * Remove dev build defs for /Zc:preprocessor and /Zc:__cplusplus. * Recheck languages on successive `find_package(libcudacxx)` calls. * Clean up dependency handling between projects. * Set <project>_DIR when locating packages with our add_subdir helper. This ensures that it behaves the same as find_package, and that later calls to find_package will locate the same configs. * Update test_packaging.sh to use the local repo for CPM. * Enable CUDA language and verbose logging for CMake example configs. * Reduce verbosity of libcudacxx package. * Fix early return checks. * Improve some option names / diagnostics. * Remove check for unsupported CTK versions.

alliepiper added 2 commits November 7, 2025 20:04

Remove MSVC hint that is applied globally in CCCL.

c4d8321

Fix oversubscription issue with lit precompile, label hack.

04f48eb

Moves the C2H tests earlier in the config so we can add the fake dependency to lit. Explains and labels the hack we're using to avoid oversubscription.

github-project-automation bot added this to CCCL Nov 7, 2025

github-project-automation bot moved this to Todo in CCCL Nov 7, 2025

cccl-authenticator-app bot moved this from Todo to In Progress in CCCL Nov 7, 2025

alliepiper commented Nov 7, 2025

View reviewed changes

alliepiper marked this pull request as ready for review November 7, 2025 20:11

alliepiper requested a review from a team as a code owner November 7, 2025 20:11

alliepiper requested a review from robertmaynard November 7, 2025 20:11

cccl-authenticator-app bot moved this from In Progress to In Review in CCCL Nov 7, 2025

This comment has been minimized.

Sign in to view

miscco previously requested changes Nov 10, 2025

View reviewed changes

github-project-automation bot moved this from In Review to In Progress in CCCL Nov 10, 2025

Merge remote-tracking branch 'origin/main' into libcudacxx-lit-fix

6396df4

alliepiper requested a review from miscco November 17, 2025 13:52

bernhardmgruber approved these changes Nov 18, 2025

View reviewed changes

alliepiper merged commit 7625cff into NVIDIA:main Nov 18, 2025
104 checks passed

alliepiper deleted the libcudacxx-lit-fix branch November 18, 2025 18:46

github-project-automation bot moved this from In Progress to Done in CCCL Nov 18, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix oversubscription issue with lit precompile, label hack#6554

Fix oversubscription issue with lit precompile, label hack#6554
alliepiper merged 3 commits intoNVIDIA:mainfrom
alliepiper:libcudacxx-lit-fix

alliepiper commented Nov 7, 2025

Uh oh!

copy-pr-bot bot commented Nov 7, 2025

Uh oh!

alliepiper Nov 7, 2025

Uh oh!

This comment has been minimized.

miscco Nov 10, 2025

Uh oh!

alliepiper Nov 10, 2025

Uh oh!

alliepiper Nov 10, 2025

Uh oh!

github-actions bot commented Nov 10, 2025

Uh oh!

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

alliepiper commented Nov 7, 2025

Uh oh!

copy-pr-bot bot commented Nov 7, 2025

Uh oh!

alliepiper Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

miscco Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

alliepiper Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

alliepiper Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Nov 10, 2025

🥳 CI Workflow Results

🟩 Finished in 1h 16m: Pass: 100%/90 | Total: 15h 09m | Max: 1h 08m | Hits: 99%/219115

Uh oh!

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants