Reduce test-suite compile time: extract tiny per-standard test content; drop redundant legacy-comparison CI job - #5481
Open
nlohmann wants to merge 2 commits into
Open
Reduce test-suite compile time: extract tiny per-standard test content; drop redundant legacy-comparison CI job#5481nlohmann wants to merge 2 commits into
nlohmann wants to merge 2 commits into
Conversation
unit-items.cpp is a 1433-line file that was being compiled twice per
CI configuration (once for C++11, once for C++17) purely because it
contained a single, small JSON_HAS_CPP_17-gated SECTION ("structured
bindings", 14 lines). Move that SECTION into a new, dedicated file
(tests/src/unit-items-cpp17.cpp) so only that tiny file needs a
second build; unit-items.cpp itself now builds/tests only once. No
tests/CMakeLists.txt changes are needed since the existing
file(GLOB ... src/unit-*.cpp) plus json_test_add_test_for() already
auto-register and standard-gate any new unit-*.cpp file based on
whether it textually contains JSON_HAS_CPP_<N> (the same mechanism
already used for the existing unit-iterators3.cpp file, which follows
the identical pattern).
Verified with plain clang++ under -std=c++11/14/17/20 and via a local
CMake configure+build that:
- unit-items.cpp now only produces a test-items_cpp11 target (the
former test-items_cpp17 target is gone) and its assertion/test-case
counts are unchanged (2 test cases / 222 assertions) for every
standard.
- The new unit-items-cpp17.cpp produces test-items-cpp17_cpp11 (an
intentionally empty translation unit under C++11 that reports 0
tests, 0 assertions, SUCCESS) and test-items-cpp17_cpp17 (1 test
case / 1 assertion, identical to what "structured bindings" ran
as before it was moved).
Separately, unit-regression1.cpp (1530 lines) was also being built
twice per CI configuration because it contained the substring
JSON_HAS_CPP_17 -- but on inspection this was dead code: an orphaned
"#ifdef JSON_HAS_CPP_17 / #include <variant> / #endif" left over from
when the actual std::variant-based regression test (issue #1292) was
relocated to unit-regression2.cpp. Nothing in unit-regression1.cpp
uses <variant>, so there is no SECTION/TEST_CASE to preserve here;
the dead include is simply removed. This was verified by grepping the
file for any other use of "variant" (none) and confirming issue #1292
is still covered by unit-regression2.cpp. Compiled and ran under
-std=c++11/14/17/20 and via CMake: unit-regression1.cpp now only
produces a test-regression1_cpp11 target (test-regression1_cpp17 is
gone) with an unchanged test-case count (3) under every standard.
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
This repo's astyle style keeps preprocessor directives at column 0 even inside #ifdef blocks. The new tests/src/unit-items-cpp17.cpp had its #include <map>/#include <string> indented, which made the 'check' CI job's amalgamation/formatting diff non-empty and failed the aggregate check. Signed-off-by: Niels Lohmann <mail@nlohmann.me>
🔴 Amalgamation check failed! 🔴The source code has not been amalgamated and/or formatted correctly. 📎 A ready-to-apply patch is attached to the failed workflow run as the git apply amalgamation.patchThis does not require installing astyle yourself. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This addresses two of the smallest, lowest-risk items from the larger compile-time audit in #5419. It does not attempt the intra-file de-duplication items from that issue (the
unit-items.cppiterator_wrapper/items() duplication, unicode helper de-dup, iterators1/2 templating, binary-format corpus list sharing, etc.) — those are left for a future PR.Fixes #5419 (partially — see "Out of scope" below for what is intentionally not done here).
Item 1 (partial): stop rebuilding two large files just for a few C++17-gated lines
tests/src/unit-items.cpp(1433 lines) was compiled twice per CI configuration (test-items_cpp11andtest-items_cpp17) solely because it contained one smallJSON_HAS_CPP_17-gatedSECTION("structured bindings")(14 lines). That section is moved into a new, dedicated file,tests/src/unit-items-cpp17.cpp. Notests/CMakeLists.txtchanges were needed: the existingfile(GLOB ... src/unit-*.cpp)+json_test_add_test_for()machinery already auto-registers anyunit-*.cppfile and auto-detects which C++ standards to build it for by checking whether the file text containsJSON_HAS_CPP_<N>— exactly the same mechanism already used by the existingtests/src/unit-iterators3.cpp(a small file dedicated to oneJSON_HAS_CPP_14-gated test), which the new file mirrors.Result:
unit-items.cppnow only producestest-items_cpp11(thetest-items_cpp17target is gone). The newunit-items-cpp17.cppproducestest-items-cpp17_cpp11(compiles to an intentionally empty translation unit under C++11 — 0 tests, 0 assertions, still a clean pass) andtest-items-cpp17_cpp17(1 test case / 1 assertion, identical in behavior to what ran before).tests/src/unit-regression1.cpp(1530 lines) was likewise built twice (test-regression1_cpp11andtest-regression1_cpp17) because it contained the substringJSON_HAS_CPP_17. On inspection, this turned out to be dead code: an orphanedleft over from when the actual
std::variant-based regression test (issue Serializing std::variant causes stack overflow #1292, "Serializing std::variant causes stack overflow") was relocated tounit-regression2.cppin an earlier split —unit-regression2.cppstill contains thatSECTIONand#include <variant>today. Nothing else inunit-regression1.cppreferencesvariant, so there was no test content to extract; the dead include is simply deleted.unit-regression1.cppnow only producestest-regression1_cpp11.Both changes were verified with
clang++directly (-std=c++11/14/17/20, doctest test-case/assertion counts compared before/after — unchanged in every case) and with a real local CMake configure + build confirming the exact target lists described above.Item 2:
ci_test_legacycomparison— investigated, not removedThe issue suggested
ci_test_legacycomparison(.github/workflows/ubuntu.yml, defined incmake/ci.cmake) is redundant with thetest-comparison_legacytarget thattests/CMakeLists.txtalready builds in every normal configuration (both useJSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1).On closer inspection, they are not redundant:
test-comparison_legacy(intests/CMakeLists.txt) applies the define as a per-targetCOMPILE_DEFINITIONSoverride to exactly one translation unit:unit-comparison.cpp. Every other test file in the suite is still compiled with the default (non-legacy) comparison semantics.ci_test_legacycomparison(incmake/ci.cmake) instead configures the whole build with-DJSON_LegacyDiscardedValueComparison=ON, which (viaCMakeLists.txt) addsJSON_USE_LEGACY_DISCARDED_VALUE_COMPARISON=1as anINTERFACEcompile definition on the mainnlohmann_jsontarget itself. Because every test target links against that target (transitively throughtest_main), this define propagates to everyunit-*.cppfile in the entire suite, across all default C++ standards. SinceJSON_USE_LEGACY_DISCARDED_VALUE_COMPARISONchanges the behavior ofoperator==for discarded values insidejson.hppitself (seeinclude/nlohmann/json.hpp), this exercises the legacy code path across the whole test suite (everyCHECK/comparison in every file), not just the handful of comparisons inunit-comparison.cpp.So
ci_test_legacycomparisonprovides meaningfully broader coverage thantest-comparison_legacy(whole-suite exercise of the legacy macro vs. one file), and I left it in place rather than removing it, per the issue's own caveat about not removing jobs that turn out to do something meaningfully different.Breaking change?
No. This PR only touches files under
tests/(test-infrastructure only); nothing underinclude/was changed, so there is no change to the public API and no ABI/API impact.No test behavior or coverage was removed — every
TEST_CASE/SECTIONthat existed before this PR still exists and still runs under the exact same C++-standard gating as before, just physically located in a different.cppfile (unit-items-cpp17.cpp) in thestructured bindingscase, or removed as genuinely dead/unused code in theunit-regression1.cpp<variant>-include case (confirmed viagit log --followthat the corresponding test itself lives on, unchanged, inunit-regression2.cpp).Out of scope (left for a future PR)
Per the issue's larger audit, none of the following are attempted here:
unit-items.cppiterator_wrapper/items()intra-file duplication.unit-regression2.cpp,unit-deserialization.cpp,unit-conversions.cpp,unit-element_access2.cpp,unit-msgpack.cpp(their gated portions are much larger and extracting them safely needs more care than fits this PR).iterators1/iterators2templating, and binary-format corpus list sharing.Test plan
clang++ -std=c++11/14/17/20compiledunit-regression1.cpp,unit-items.cpp, and the newunit-items-cpp17.cppcleanly.unit-regression1.cpp: 3 test cases under all of 11/14/17/20;unit-items.cpp: 2 test cases / 222 assertions under all of 11/14/17/20;unit-items-cpp17.cpp: 0 tests under 11, 1 test case / 1 assertion under 17/20).origin/developcopy ofunit-regression1.cppthat an unrelated, pre-existing local-environment failure (issue Parse throw std::ios_base::failure exception when failbit set to true #714,std::ios_base::failurebehavior under this machine's libc++) is present identically before and after this change, i.e. not introduced by it.-DJSON_BuildTests=ON) +cmake --build . --target helpconfirmed the exact expected target-list changes:test-regression1_cpp17andtest-items_cpp17no longer exist;test-items-cpp17_cpp11andtest-items-cpp17_cpp17now exist and build/pass.— opened by Claude Code on behalf of @nlohmann