Skip to content

Test structure is stressful in terms of compiler memory consumption #321

@StephanTLavavej

Description

@StephanTLavavej

Hi Boost.Parser devs! I'm the primary maintainer of MSVC's STL, and our compiler team has encountered issues with recent MSVC STL changes (specifically our implementation of C++23 reference_meows_from_temporary in tuple's constructors) causing Boost.Parser tests to have excessive compiler memory consumption, reaching out-of-memory for our accursed 32-bit x86-hosted compiler. (We regularly test open-source projects, especially Boost, with development builds of the compiler and libraries to find and fix regressions before they can ship and affect you.)

This is not a bug in your tests so you can close this issue if you like. However, I wanted to at least mention that the specific structure of the tests is leading to this excessive compiler memory consumption and a simple refactoring would make them less likely to OOM, and (as a benefit to you) probably a bit faster to compile.

The issue is when main() or any function has a long list of static_asserts and instantiations like this:

{
constexpr auto parser = merge[char_ >> ' ' >> char_];
static_assert(
std::is_same_v<decltype(parse("", parser)), std::optional<char>>);
{
auto result = parse("a b", parser);
BOOST_TEST(result);
BOOST_TEST(*result == 'b');
}
}
{
constexpr auto parser = char_ >> merge[char_ >> char_] >> char_;
static_assert(std::is_same_v<
decltype(parse("", parser)),
std::optional<tuple<char, char, char>>>);
{
BOOST_TEST(!parse("ab", parser));
BOOST_TEST(!parse("abc", parser));
auto result = parse("abcd", parser);
BOOST_TEST(result);
BOOST_TEST(*result == detail::hl::make_tuple('a', 'c', 'd'));
}
}

Even though this code is in separate scopes, they're still within the same function, so the compiler doesn't reuse memory effectively. If these were in separate functions, then I believe the compiler memory consumption would be limited. (I don't have an exhaustive list of tests affected, this is just the one in the error message I saw.)

If you're interested in doing this, the undocumented but very useful compiler option /d1reportMemorySummary will report memory consumption. Here's an example showing how <print> is a chonker compared to <cstdio>, "Peak working set size" is the value of interest:

C:\Temp>type meow.cpp
#ifdef USE_CXX23
#include <print>
#else
#include <cstdio>
#endif

int main() {
#ifdef USE_CXX23
    std::println("Hello C++23 world!");
#else
    std::puts("Hello C++98 world!");
#endif
}

C:\Temp>cl /EHsc /nologo /W4 /MTd /Od /std:c++14 /d1reportMemorySummary meow.cpp && meow
meow.cpp
Persistent heap 1 size: 5424544
Persistent heap 2 size: 1255624
Peak working set size: 20422656
Hello C++98 world!

C:\Temp>cl /EHsc /nologo /W4 /MTd /Od /std:c++latest /DUSE_CXX23 /d1reportMemorySummary meow.cpp && meow
meow.cpp
Persistent heap 1 size: 84938280
Persistent heap 2 size: 4237816
Peak working set size: 113741824
Hello C++23 world!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions