Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Replace Catch with Doctest #1921

Merged
merged 1 commit into from
Dec 24, 2021
Merged

Replace Catch with Doctest #1921

merged 1 commit into from
Dec 24, 2021

Conversation

vittorioromeo
Copy link
Member

@vittorioromeo vittorioromeo commented Dec 21, 2021 •
edited

This PR changes the testing framework from Catch to doctest, mostly to improve compilation times and reduce the load on CI machines. The results speak for themselves:

# `time ninja` with catch (fresh build folder)
ninja  0.00s user 0.01s system 0% cpu 21.935 total

# `time ninja` with doctest (fresh build folder)
ninja  0.00s user 0.00s system 0% cpu 17.927 total
Compilation time analysis with Catch 
Analyzing build trace from 'analysis.txt'...
**** Time summary:
Compilation (225 times):
  Parsing (frontend):          149.8 s
  Codegen & opts (backend):     71.3 s

**** Files that took longest to parse (compiler frontend):
  4903 ms: test/CMakeFiles/test-sfml-system.dir/src/CatchMain.cpp.obj
  4295 ms: test/CMakeFiles/test-sfml-window.dir/src/CatchMain.cpp.obj
  4020 ms: test/CMakeFiles/sfml-test-main.dir/src/CatchMain.cpp.obj
  3678 ms: test/CMakeFiles/test-sfml-graphics.dir/src/CatchMain.cpp.obj
  3359 ms: test/CMakeFiles/test-sfml-network.dir/src/CatchMain.cpp.obj
  3216 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/WindowImpl.cpp.obj
  2675 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/GlContext.cpp.obj
  2624 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/EglContext.cpp.obj
  2418 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/Win32/JoystickImpl.cpp.obj
  2349 ms: test/CMakeFiles/test-sfml-system.dir/src/System/Vector2.cpp.obj

**** Files that took longest to codegen (compiler backend):
  8046 ms: src/SFML/Graphics/CMakeFiles/sfml-graphics.dir/ImageLoader.cpp.obj
  6334 ms: test/CMakeFiles/test-sfml-window.dir/src/CatchMain.cpp.obj
  6235 ms: test/CMakeFiles/test-sfml-system.dir/src/CatchMain.cpp.obj
  6216 ms: test/CMakeFiles/test-sfml-graphics.dir/src/CatchMain.cpp.obj
  6137 ms: test/CMakeFiles/test-sfml-network.dir/src/CatchMain.cpp.obj
  4759 ms: test/CMakeFiles/sfml-test-main.dir/src/CatchMain.cpp.obj
  3071 ms: examples/opengl/CMakeFiles/opengl.dir/OpenGL.cpp.obj
  2330 ms: examples/window/CMakeFiles/window.dir/Window.cpp.obj
  2057 ms: src/SFML/Graphics/CMakeFiles/sfml-graphics.dir/GLExtensions.cpp.obj
  1964 ms: src/SFML/Audio/CMakeFiles/sfml-audio.dir/SoundFileReaderMp3.cpp.obj

**** Templates that took longest to instantiate:
   750 ms: std::basic_string<char16_t>::_M_construct<const char16_t *> (186 times, avg 4 ms)
   732 ms: std::basic_string<char32_t>::_M_construct<const char32_t *> (186 times, avg 3 ms)
   705 ms: std::unordered_map<int, int> (39 times, avg 18 ms)
   598 ms: std::_Hashtable<int, std::pair<const int, int>, std::allocator<std::... (39 times, avg 15 ms)
   594 ms: __gnu_cxx::__to_xstring<std::basic_string<wchar_t>, wchar_t> (93 times, avg 6 ms)
   506 ms: std::basic_string<wchar_t>::_M_construct<wchar_t *> (186 times, avg 2 ms)
   497 ms: std::basic_string<char32_t> (93 times, avg 5 ms)
   487 ms: std::basic_string<char32_t>::basic_string (93 times, avg 5 ms)
   471 ms: std::basic_string<wchar_t> (93 times, avg 5 ms)
   469 ms: std::basic_string<char16_t>::basic_string (93 times, avg 5 ms)
   446 ms: std::basic_string<char16_t> (93 times, avg 4 ms)
   444 ms: __gnu_cxx::__to_xstring<std::basic_string<char>, char> (93 times, avg 4 ms)
   415 ms: std::basic_string<char> (93 times, avg 4 ms)
   378 ms: std::basic_string<char16_t>::_M_construct_aux<const char16_t *> (93 times, avg 4 ms)
   364 ms: std::basic_string<wchar_t>::basic_string<wchar_t *, void> (93 times, avg 3 ms)
   359 ms: std::basic_string<char32_t>::_M_construct_aux<const char32_t *> (93 times, avg 3 ms)
   350 ms: std::operator+<char, std::char_traits<char>, std::allocator<char>> (197 times, avg 1 ms)
   281 ms: std::basic_string<char>::_M_construct<char *> (185 times, avg 1 ms)
   274 ms: std::__and_<std::is_convertible<const char16_t *const &, std::basic_... (93 times, avg 2 ms)
   256 ms: std::basic_string_view<char> (93 times, avg 2 ms)
   254 ms: std::__detail::__hyperg<float> (25 times, avg 10 ms)
   251 ms: std::basic_string<wchar_t>::_M_construct_aux<wchar_t *> (93 times, avg 2 ms)
   243 ms: std::__detail::_Insert<int, std::pair<const int, int>, std::allocato... (39 times, avg 6 ms)
   232 ms: std::basic_string<char>::basic_string (190 times, avg 1 ms)
   230 ms: std::unordered_multimap<int, int> (39 times, avg 5 ms)
   223 ms: std::__detail::_Insert_base<int, std::pair<const int, int>, std::all... (39 times, avg 5 ms)
   214 ms: std::__and_<std::is_convertible<const char32_t *const &, std::basic_... (93 times, avg 2 ms)
   211 ms: std::__detail::__hyperg<long double> (25 times, avg 8 ms)
   207 ms: std::unique_ptr<std::thread::_State> (11 times, avg 18 ms)
   205 ms: std::pair<bool, unsigned long long> (45 times, avg 4 ms)

**** Template sets that took longest to instantiate:
  3270 ms: std::__and_<$> (2201 times, avg 1 ms)
  1992 ms: std::basic_string<$> (405 times, avg 4 ms)
  1492 ms: std::_Hashtable<$> (147 times, avg 10 ms)
  1431 ms: std::basic_string<$>::_M_construct<$> (565 times, avg 2 ms)
  1312 ms: std::basic_string<$>::basic_string (474 times, avg 2 ms)
  1310 ms: std::basic_string<$>::_M_construct_aux<$> (559 times, avg 2 ms)
  1299 ms: std::unordered_map<$> (81 times, avg 16 ms)
  1232 ms: __gnu_cxx::__stoa<$> (1442 times, avg 0 ms)
  1226 ms: std::__or_<$> (1386 times, avg 0 ms)
  1103 ms: std::is_convertible<$> (1239 times, avg 0 ms)
  1038 ms: __gnu_cxx::__to_xstring<$> (186 times, avg 5 ms)
   904 ms: std::vector<$>::push_back (141 times, avg 6 ms)
   803 ms: std::pair<$> (267 times, avg 3 ms)
   799 ms: std::basic_string_view<$> (375 times, avg 2 ms)
   746 ms: std::vector<$>::_M_realloc_insert<$> (146 times, avg 5 ms)
   718 ms: std::vector<$> (260 times, avg 2 ms)
   656 ms: std::basic_string<$>::basic_string<$> (283 times, avg 2 ms)
   623 ms: std::__not_<$> (662 times, avg 0 ms)
   589 ms: std::chrono::duration<$> (441 times, avg 1 ms)
   571 ms: std::__detail::_Insert<$> (127 times, avg 4 ms)
   506 ms: std::unique_ptr<$> (26 times, avg 19 ms)
   503 ms: std::__detail::_Insert_base<$> (103 times, avg 4 ms)
   478 ms: std::is_nothrow_default_constructible<$> (404 times, avg 1 ms)
   466 ms: std::__detail::__hyperg<$> (50 times, avg 9 ms)
   420 ms: std::is_trivial<$> (377 times, avg 1 ms)
   418 ms: std::__uniq_ptr_data<$> (26 times, avg 16 ms)
   411 ms: std::__detail::_Hashtable_alloc<$> (99 times, avg 4 ms)
   403 ms: std::__uniq_ptr_impl<$> (26 times, avg 15 ms)
   369 ms: std::vector<$>::_S_use_relocate (155 times, avg 2 ms)
   369 ms: std::vector<$>::~vector (165 times, avg 2 ms)

**** Functions that took longest to compile:
   980 ms: main (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
   514 ms: ____C_A_T_C_H____T_E_S_T____0() (C:/OHW/SFML/test/src/System/Vector3.cpp)
   514 ms: gladLoadGLUserPtr(void (* (*)(void*, char const*))(), void*) (C:/OHW/SFML/src/SFML/Graphics/GLExtensions.cpp)
   493 ms: Catch::makeCommandLineParser() (C:/OHW/SFML/test/src/CatchMain.cpp)
   440 ms: ____C_A_T_C_H____T_E_S_T____0() (C:/OHW/SFML/test/src/System/Vector2.cpp)
   402 ms: mp3dec_decode_frame (C:/OHW/SFML/src/SFML/Audio/SoundFileReaderMp3.cpp)
   372 ms: stbi__jpeg_load(stbi__context*, int*, int*, int*, int, stbi__result_... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   356 ms: ____C_A_T_C_H____T_E_S_T____0() (C:/OHW/SFML/test/src/Graphics/Rect.cpp)
   289 ms: main (C:/OHW/SFML/examples/window/Window.cpp)
   270 ms: stbi__create_png_image_raw(stbi__png*, unsigned char*, unsigned int,... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   254 ms: gladLoadGLUserPtr (C:/OHW/SFML/examples/window/Window.cpp)
   235 ms: glad_gl_resolve_aliases() (C:/OHW/SFML/examples/window/Window.cpp)
   209 ms: gladLoadGL (C:/OHW/SFML/examples/window/Window.cpp)
   199 ms: stbi__load_main(stbi__context*, int*, int*, int*, int, stbi__result_... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   187 ms: Catch::ConsoleReporter::printTotals(Catch::Totals const&) (C:/OHW/SFML/test/src/CatchMain.cpp)
   183 ms: Catch::listTests(Catch::Config const&) (C:/OHW/SFML/test/src/CatchMain.cpp)
   172 ms: Catch::listTags(Catch::Config const&) (C:/OHW/SFML/test/src/CatchMain.cpp)
   138 ms: gladLoadGLUserPtr (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
   128 ms: stbi__tga_load(stbi__context*, int*, int*, int*, int, stbi__result_i... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   126 ms: main (C:/OHW/SFML/examples/ftp/Ftp.cpp)
   122 ms: Catch::XmlReporter::assertionEnded(Catch::AssertionStats const&) (C:/OHW/SFML/test/src/CatchMain.cpp)
   120 ms: stbi__do_zlib(stbi__zbuf*, char*, int, int, int) (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   113 ms: main (C:/OHW/SFML/examples/joystick/Joystick.cpp)
   110 ms: Catch::CompactReporter::printTotals(Catch::Totals const&) const (C:/OHW/SFML/test/src/CatchMain.cpp)
   109 ms: stbi__hdr_load(stbi__context*, int*, int*, int*, int, stbi__result_i... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   108 ms: Catch::Tbc::Text::Text(std::__cxx11::basic_string<char, std::char_tr... (C:/OHW/SFML/test/src/CatchMain.cpp)
   106 ms: stbiw__encode_png_line(unsigned char*, int, int, int, int, int, int,... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   104 ms: gladLoadGL (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
    96 ms: Tbc::Text::Text(std::__cxx11::basic_string<char, std::char_traits<ch... (C:/OHW/SFML/test/src/CatchMain.cpp)
    93 ms: Catch::Clara::CommandLine<Catch::ConfigData>::populateOptions(std::v... (C:/OHW/SFML/test/src/CatchMain.cpp)

**** Function sets that took longest to compile / optimize:
   276 ms: void std::vector<$>::_M_realloc_insert<$>(__gnu_cxx::__normal_iterat... (8 times, avg 34 ms)
   193 ms: std::vector<$>::_M_default_append(unsigned long long) (24 times, avg 8 ms)
   162 ms: void Catch::TestSpecParser::addPattern<$>() (2 times, avg 81 ms)
   123 ms: Catch::BinaryExpression<$>::reconstructExpression(std::__cxx11::basi... (14 times, avg 8 ms)
   108 ms: Catch::Tbc::Text::Text(std::__cxx11::basic_string<$> const&, Catch::... (1 times, avg 108 ms)
    96 ms: Tbc::Text::Text(std::__cxx11::basic_string<$> const&, Tbc::TextAttri... (1 times, avg 96 ms)
    93 ms: Catch::Clara::CommandLine<$>::populateOptions(std::vector<$> const&,... (1 times, avg 93 ms)
    90 ms: Catch::toString(std::__cxx11::basic_string<$> const&) (2 times, avg 45 ms)
    90 ms: std::_Rb_tree<$>::_M_get_insert_unique_pos(std::__cxx11::basic_strin... (7 times, avg 12 ms)
    89 ms: std::pair<$> std::_Rb_tree<$>::_M_emplace_unique<$>(std::pair<$>&&) (5 times, avg 17 ms)
    87 ms: Catch::JunitReporter::writeSection(std::__cxx11::basic_string<$> con... (1 times, avg 87 ms)
    85 ms: std::vector<$>::~vector() (26 times, avg 3 ms)
    82 ms: Catch::runTests(Catch::Ptr<$> const&) (1 times, avg 82 ms)
    82 ms: __gnu_cxx::__normal_iterator<$> std::__find_if<$>(__gnu_cxx::__norma... (2 times, avg 41 ms)
    80 ms: Catch::RunContext::handleFatalErrorCondition(std::__cxx11::basic_str... (1 times, avg 80 ms)
    76 ms: Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<$>&, st... (1 times, avg 76 ms)
    76 ms: Catch::JunitReporter::writeGroup(Catch::CumulativeReporterBase::Node... (1 times, avg 76 ms)
    67 ms: Catch::AssertionStats::AssertionStats(Catch::AssertionResult const&,... (1 times, avg 67 ms)
    59 ms: void std::vector<$>::_M_range_insert<$>(__gnu_cxx::__normal_iterator... (1 times, avg 59 ms)
    59 ms: Catch::loadTestNamesFromFile(Catch::ConfigData&, std::__cxx11::basic... (1 times, avg 59 ms)
    59 ms: Catch::TagAliasRegistry::add(std::__cxx11::basic_string<$> const&, s... (1 times, avg 59 ms)
    58 ms: Catch::makeTestCase(Catch::ITestCase*, std::__cxx11::basic_string<$>... (1 times, avg 58 ms)
    56 ms: std::_Rb_tree<$>::_M_erase(std::_Rb_tree_node<$>*) (11 times, avg 5 ms)
    52 ms: Catch::ConsoleReporter::printSummaryRow(std::__cxx11::basic_string<$... (1 times, avg 52 ms)
    52 ms: std::vector<$>::vector(std::vector<$> const&) (3 times, avg 17 ms)
    51 ms: Catch::TagAliasRegistry::expandAliases(std::__cxx11::basic_string<$>... (1 times, avg 51 ms)
    50 ms: Catch::CumulativeReporterBase::Node<$>::~Node() (3 times, avg 16 ms)
    50 ms: Catch::Clara::CommandLine<$>::optUsage(std::ostream&, unsigned long ... (1 times, avg 50 ms)
    47 ms: Catch::setTags(Catch::TestCaseInfo&, std::set<$> const&) (1 times, avg 47 ms)
    46 ms: Catch::replaceInPlace(std::__cxx11::basic_string<$>&, std::__cxx11::... (1 times, avg 46 ms)

*** Expensive headers:
29815 ms: C:/msys64/mingw64/x86_64-w64-mingw32/include/windows.h (included 29 times, avg 1028 ms), included via:
  MainWin32.cpp.obj WindowsHeader.hpp  (1743 ms)
  GlContext.cpp.obj EglContext.hpp egl.h  (1481 ms)
  EglContext.cpp.obj EglContext.hpp egl.h  (1454 ms)
  CatchMain.cpp.obj catch.hpp  (1157 ms)
  EGLCheck.cpp.obj egl.h  (1154 ms)
  CatchMain.cpp.obj catch.hpp  (1077 ms)
  ...

23229 ms: C:/OHW/SFML/extlibs/headers/catch.hpp (included 9 times, avg 2581 ms), included via:
  CatchMain.cpp.obj  (4114 ms)
  CatchMain.cpp.obj  (3634 ms)
  CatchMain.cpp.obj  (3444 ms)
  CatchMain.cpp.obj  (3095 ms)
  CatchMain.cpp.obj  (2731 ms)
  Vector3.cpp.obj SystemUtil.hpp  (2024 ms)
  ...

19926 ms: C:/OHW/SFML/src/SFML/System/Win32/WindowsHeader.hpp (included 20 times, avg 996 ms), included via:
  MainWin32.cpp.obj  (1744 ms)
  InputImpl.cpp.obj  (1075 ms)
  JoystickManager.cpp.obj JoystickManager.hpp JoystickImpl.hpp JoystickImpl.hpp  (1038 ms)
  SocketImpl.cpp.obj SocketImpl.hpp  (1023 ms)
  UdpSocket.cpp.obj SocketImpl.hpp SocketImpl.hpp  (1008 ms)
  TcpSocket.cpp.obj SocketImpl.hpp SocketImpl.hpp  (999 ms)
  ...

18102 ms: C:/OHW/SFML/include/SFML/System/Err.hpp (included 60 times, avg 301 ms), included via:
  Err.cpp.obj  (742 ms)
  SoundFileReaderWav.cpp.obj  (608 ms)
  Context.cpp.obj  (596 ms)
  EGLCheck.cpp.obj  (582 ms)
  VertexBuffer.cpp.obj  (545 ms)
  ALCheck.cpp.obj  (542 ms)
  ...

15939 ms: C:/msys64/mingw64/include/c++/11.2.0/bits/ios_base.h (included 84 times, avg 189 ms), included via:
  sstream istream ios  (451 ms)
  SystemUtil.hpp catch.hpp sstream istream ios  (443 ms)
  String.hpp Utf.hpp iterator streambuf_iterator.h streambuf  (413 ms)
  sstream istream ios  (411 ms)
  catch.hpp sstream istream ios  (389 ms)
  GraphicsUtil.hpp WindowUtil.hpp SystemUtil.hpp catch.hpp sstream istream ios  (380 ms)
  ...

12809 ms: C:/msys64/mingw64/include/c++/11.2.0/bits/basic_string.h (included 93 times, avg 137 ms), included via:
  SoundBufferRecorder.hpp SoundBuffer.hpp string  (286 ms)
  SystemUtil.hpp catch.hpp sstream istream ios ios_base.h locale_classes.h string  (251 ms)
  sstream istream ios ios_base.h locale_classes.h string  (230 ms)
  AudioDevice.hpp string  (222 ms)
  GraphicsUtil.hpp WindowUtil.hpp SystemUtil.hpp catch.hpp sstream istream ios ios_base.h locale_classes.h string  (214 ms)
  sstream istream ios ios_base.h locale_classes.h string  (202 ms)
  ...

11340 ms: C:/OHW/SFML/include/SFML/System/String.hpp (included 31 times, avg 365 ms), included via:
  SystemUtil.cpp.obj  (1147 ms)
  String.cpp.obj  (883 ms)
  Clipboard.cpp.obj  (780 ms)
  Joystick.cpp.obj Joystick.hpp  (729 ms)
  ClipboardImpl.cpp.obj  (657 ms)
  WindowImplWin32.cpp.obj WindowImplWin32.hpp Event.hpp Joystick.hpp  (601 ms)
  ...

8329 ms: C:/msys64/mingw64/include/c++/11.2.0/bits/stl_algobase.h (included 108 times, avg 77 ms), included via:
  SoundRecorder.hpp vector  (251 ms)
  VideoMode.hpp vector  (203 ms)
  SoundFileFactory.hpp string char_traits.h  (202 ms)
  ConvexShape.hpp Shape.hpp VertexArray.hpp vector  (144 ms)
  RectangleShape.hpp Shape.hpp VertexArray.hpp vector  (141 ms)
  cmath specfun.h  (137 ms)
  ...

8213 ms: C:/OHW/SFML/src/SFML/Network/Win32/SocketImpl.hpp (included 8 times, avg 1026 ms), included via:
  SocketImpl.cpp.obj  (1218 ms)
  SocketSelector.cpp.obj SocketImpl.hpp  (1053 ms)
  UdpSocket.cpp.obj SocketImpl.hpp  (1032 ms)
  TcpSocket.cpp.obj SocketImpl.hpp  (1023 ms)
  IpAddress.cpp.obj SocketImpl.hpp  (993 ms)
  Packet.cpp.obj SocketImpl.hpp  (982 ms)
  ...

7721 ms: C:/OHW/SFML/include/SFML/Graphics.hpp (included 7 times, avg 1103 ms), included via:
  Island.cpp.obj  (1225 ms)
  Joystick.cpp.obj  (1167 ms)
  Vulkan.cpp.obj  (1118 ms)
  Win32.cpp.obj  (1117 ms)
  OpenGL.cpp.obj  (1087 ms)
  Shader.cpp.obj Effect.hpp  (1024 ms)
  ...

  done in 0.0s.
Compilation time analysis with Doctest 
Analyzing build trace from 'analysis.txt'...
**** Time summary:
Compilation (215 times):
  Parsing (frontend):          136.4 s
  Codegen & opts (backend):     44.5 s

**** Files that took longest to parse (compiler frontend):
  3680 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/WindowImpl.cpp.obj
  3308 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/Win32/JoystickImpl.cpp.obj
  3285 ms: test/CMakeFiles/sfml-test-main.dir/src/DoctestMain.cpp.obj
  2577 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/Win32/WindowImplWin32.cpp.obj
  2476 ms: examples/win32/CMakeFiles/win32.dir/Win32.cpp.obj
  2371 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/GlContext.cpp.obj
  2299 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/Win32/WglContext.cpp.obj
  2235 ms: src/SFML/Network/CMakeFiles/sfml-network.dir/IpAddress.cpp.obj
  2186 ms: src/SFML/Window/CMakeFiles/sfml-window.dir/JoystickManager.cpp.obj
  2139 ms: src/SFML/Network/CMakeFiles/sfml-network.dir/TcpSocket.cpp.obj

**** Files that took longest to codegen (compiler backend):
  8104 ms: src/SFML/Graphics/CMakeFiles/sfml-graphics.dir/ImageLoader.cpp.obj
  3497 ms: examples/opengl/CMakeFiles/opengl.dir/OpenGL.cpp.obj
  2291 ms: test/CMakeFiles/sfml-test-main.dir/src/DoctestMain.cpp.obj
  2074 ms: examples/window/CMakeFiles/window.dir/Window.cpp.obj
  1946 ms: src/SFML/Graphics/CMakeFiles/sfml-graphics.dir/GLExtensions.cpp.obj
  1805 ms: src/SFML/Audio/CMakeFiles/sfml-audio.dir/SoundFileReaderMp3.cpp.obj
  1684 ms: examples/vulkan/CMakeFiles/vulkan.dir/Vulkan.cpp.obj
   920 ms: src/SFML/Network/CMakeFiles/sfml-network.dir/Ftp.cpp.obj
   906 ms: src/SFML/Graphics/CMakeFiles/sfml-graphics.dir/Shader.cpp.obj
   902 ms: examples/island/CMakeFiles/island.dir/Island.cpp.obj

**** Templates that took longest to instantiate:
   640 ms: std::basic_string<char16_t>::_M_construct<const char16_t *> (178 times, avg 3 ms)
   615 ms: std::basic_string<char32_t>::_M_construct<const char32_t *> (178 times, avg 3 ms)
   591 ms: __gnu_cxx::__to_xstring<std::basic_string<wchar_t>, wchar_t> (89 times, avg 6 ms)
   550 ms: std::unordered_map<int, int> (31 times, avg 17 ms)
   548 ms: std::basic_string<wchar_t>::_M_construct<wchar_t *> (178 times, avg 3 ms)
   513 ms: std::basic_string<char16_t> (89 times, avg 5 ms)
   504 ms: std::basic_string<wchar_t> (89 times, avg 5 ms)
   494 ms: std::basic_string<char32_t> (89 times, avg 5 ms)
   471 ms: std::_Hashtable<int, std::pair<const int, int>, std::allocator<std::... (31 times, avg 15 ms)
   427 ms: std::basic_string<char16_t>::basic_string (89 times, avg 4 ms)
   419 ms: std::basic_string<char> (89 times, avg 4 ms)
   416 ms: __gnu_cxx::__to_xstring<std::basic_string<char>, char> (89 times, avg 4 ms)
   396 ms: std::basic_string<char32_t>::basic_string (89 times, avg 4 ms)
   371 ms: std::basic_string<wchar_t>::basic_string<wchar_t *, void> (89 times, avg 4 ms)
   323 ms: std::basic_string<char16_t>::_M_construct_aux<const char16_t *> (89 times, avg 3 ms)
   313 ms: std::basic_string<char32_t>::_M_construct_aux<const char32_t *> (89 times, avg 3 ms)
   303 ms: std::__and_<std::is_convertible<const wchar_t *const &, std::basic_s... (89 times, avg 3 ms)
   292 ms: std::operator+<char, std::char_traits<char>, std::allocator<char>> (176 times, avg 1 ms)
   277 ms: std::basic_string<char>::basic_string (184 times, avg 1 ms)
   274 ms: std::basic_string<wchar_t>::_M_construct_aux<wchar_t *> (89 times, avg 3 ms)
   261 ms: std::basic_string_view<wchar_t> (89 times, avg 2 ms)
   239 ms: std::basic_string_view<char> (89 times, avg 2 ms)
   237 ms: std::basic_string<char>::_M_construct<char *> (178 times, avg 1 ms)
   233 ms: std::pair<bool, unsigned long long> (37 times, avg 6 ms)
   230 ms: std::unordered_multimap<int, int> (31 times, avg 7 ms)
   224 ms: std::__and_<std::is_convertible<const char16_t *const &, std::basic_... (89 times, avg 2 ms)
   203 ms: std::basic_string<char>::_M_construct<const char *> (177 times, avg 1 ms)
   200 ms: std::__detail::__hyperg<long double> (17 times, avg 11 ms)
   197 ms: std::_Hashtable<int, std::pair<const int, int>, std::allocator<std::... (31 times, avg 6 ms)
   196 ms: std::__and_<std::is_convertible<const char32_t *const &, std::basic_... (89 times, avg 2 ms)

**** Template sets that took longest to instantiate:
  2749 ms: std::__and_<$> (1623 times, avg 1 ms)
  2108 ms: std::basic_string<$> (389 times, avg 5 ms)
  1439 ms: std::_Hashtable<$> (131 times, avg 10 ms)
  1289 ms: std::basic_string<$>::_M_construct<$> (536 times, avg 2 ms)
  1240 ms: std::basic_string<$>::basic_string (456 times, avg 2 ms)
  1225 ms: __gnu_cxx::__stoa<$> (1388 times, avg 0 ms)
  1209 ms: std::basic_string<$>::_M_construct_aux<$> (535 times, avg 2 ms)
  1208 ms: std::unordered_map<$> (73 times, avg 16 ms)
  1076 ms: std::is_convertible<$> (1044 times, avg 1 ms)
  1061 ms: std::__or_<$> (954 times, avg 1 ms)
  1008 ms: __gnu_cxx::__to_xstring<$> (178 times, avg 5 ms)
   856 ms: std::basic_string_view<$> (359 times, avg 2 ms)
   703 ms: std::__not_<$> (581 times, avg 1 ms)
   662 ms: std::basic_string<$>::basic_string<$> (268 times, avg 2 ms)
   518 ms: std::pair<$> (148 times, avg 3 ms)
   518 ms: std::chrono::duration<$> (449 times, avg 1 ms)
   517 ms: std::__detail::_Insert<$> (107 times, avg 4 ms)
   498 ms: std::is_trivial<$> (360 times, avg 1 ms)
   491 ms: std::__detail::_Insert_base<$> (94 times, avg 5 ms)
   459 ms: std::vector<$> (147 times, avg 3 ms)
   368 ms: std::__detail::_Hashtable_alloc<$> (91 times, avg 4 ms)
   358 ms: std::__detail::__hyperg<$> (34 times, avg 10 ms)
   349 ms: std::basic_string<$>::_M_dispose (374 times, avg 0 ms)
   316 ms: std::is_nothrow_default_constructible<$> (230 times, avg 1 ms)
   292 ms: std::operator+<char, std::char_traits<char>, std::allocator<char>> (176 times, avg 1 ms)
   262 ms: std::vector<$>::push_back (30 times, avg 8 ms)
   261 ms: std::__detail::__hyperg_reflect<$> (34 times, avg 7 ms)
   257 ms: __gnu_cxx::__numeric_traits_integer<$> (333 times, avg 0 ms)
   243 ms: std::vector<$>::_M_realloc_insert<$> (38 times, avg 6 ms)
   233 ms: std::__detail::__to_chars_len<$> (252 times, avg 0 ms)

**** Functions that took longest to compile:
  1217 ms: main (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
   497 ms: gladLoadGLUserPtr(void (* (*)(void*, char const*))(), void*) (C:/OHW/SFML/src/SFML/Graphics/GLExtensions.cpp)
   442 ms: stbi__create_png_image_raw(stbi__png*, unsigned char*, unsigned int,... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   318 ms: main (C:/OHW/SFML/examples/window/Window.cpp)
   277 ms: stbi__jpeg_load(stbi__context*, int*, int*, int*, int, stbi__result_... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   270 ms: DOCTEST_ANON_FUNC_2() (C:/OHW/SFML/test/src/System/Vector3.cpp)
   264 ms: mp3dec_decode_frame (C:/OHW/SFML/src/SFML/Audio/SoundFileReaderMp3.cpp)
   217 ms: gladLoadGLUserPtr (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
   202 ms: DOCTEST_ANON_FUNC_2() (C:/OHW/SFML/test/src/System/Vector2.cpp)
   200 ms: glad_gl_resolve_aliases() (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
   192 ms: glad_gl_resolve_aliases() (C:/OHW/SFML/examples/window/Window.cpp)
   186 ms: gladLoadGLUserPtr (C:/OHW/SFML/examples/window/Window.cpp)
   178 ms: stbi__load_main(stbi__context*, int*, int*, int*, int, stbi__result_... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   172 ms: gladLoadGL (C:/OHW/SFML/examples/opengl/OpenGL.cpp)
   169 ms: gladLoadGL (C:/OHW/SFML/examples/window/Window.cpp)
   162 ms: DOCTEST_ANON_FUNC_2() (C:/OHW/SFML/test/src/Graphics/Rect.cpp)
   153 ms: main (C:/OHW/SFML/examples/ftp/Ftp.cpp)
   124 ms: stbi__do_zlib(stbi__zbuf*, char*, int, int, int) (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   104 ms: stbiw__encode_png_line(unsigned char*, int, int, int, int, int, int,... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   101 ms: stbi__tga_load(stbi__context*, int*, int*, int*, int, stbi__result_i... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
   100 ms: sf::priv::WglContext::createContext(sf::priv::WglContext*) (C:/OHW/SFML/src/SFML/Window/Win32/WglContext.cpp)
    99 ms: main (C:/OHW/SFML/examples/joystick/Joystick.cpp)
    99 ms: threadFunction() (C:/OHW/SFML/examples/island/Island.cpp)
    99 ms: main (C:/OHW/SFML/examples/island/Island.cpp)
    99 ms: stbi__parse_png_file(stbi__png*, int, int) (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
    88 ms: sf::Ftp::getResponse() (C:/OHW/SFML/src/SFML/Network/Ftp.cpp)
    87 ms: stbi_write_jpg_core(stbi__write_context*, int, int, int, void const*... (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)
    87 ms: main (C:/OHW/SFML/examples/tennis/Tennis.cpp)
    84 ms: mp3d_synth_granule(float*, float*, int, int, short*, float*) (C:/OHW/SFML/src/SFML/Audio/SoundFileReaderMp3.cpp)
    79 ms: stbi_zlib_compress (C:/OHW/SFML/src/SFML/Graphics/ImageLoader.cpp)

**** Function sets that took longest to compile / optimize:
   176 ms: std::vector<$>::_M_default_append(unsigned long long) (24 times, avg 7 ms)
    49 ms: sf::Ftp::sendCommand(std::__cxx11::basic_string<$> const&, std::__cx... (1 times, avg 49 ms)
    48 ms: sf::SoundFileFactory::createReaderFromFilename(std::__cxx11::basic_s... (1 times, avg 48 ms)
    45 ms: std::_Hashtable<$>::_M_insert_unique_node(unsigned long long, unsign... (9 times, avg 5 ms)
    42 ms: std::_Hashtable<$>::_M_rehash_aux(unsigned long long, std::integral_... (9 times, avg 4 ms)
    41 ms: void std::vector<$>::_M_assign_aux<$>(short const*, short const*, st... (2 times, avg 20 ms)
    38 ms: sf::SoundFileFactory::createWriterFromFilename(std::__cxx11::basic_s... (1 times, avg 38 ms)
    35 ms: void std::_Hashtable<$>::_M_assign<$>(std::_Hashtable<$> const&, std... (3 times, avg 11 ms)
    33 ms: sf::Ftp::download(std::__cxx11::basic_string<$> const&, std::__cxx11... (1 times, avg 33 ms)
    28 ms: std::unordered_map<$>::~unordered_map() (5 times, avg 5 ms)
    27 ms: sf::Ftp::upload(std::__cxx11::basic_string<$> const&, std::__cxx11::... (1 times, avg 27 ms)
    26 ms: sf::Ftp::Response::Response(sf::Ftp::Response::Status, std::__cxx11:... (1 times, avg 26 ms)
    26 ms: std::vector<$>::~vector() (20 times, avg 1 ms)
    23 ms: sf::Shader::setUniformArray(std::__cxx11::basic_string<$> const&, sf... (2 times, avg 11 ms)
    23 ms: playMusic(std::__cxx11::basic_string<$> const&) (1 times, avg 23 ms)
    21 ms: sf::Shader::loadFromFile(std::__cxx11::basic_string<$> const&, std::... (1 times, avg 21 ms)
    21 ms: sf::Http::Response::parse(std::__cxx11::basic_string<$> const&) (1 times, avg 21 ms)
    20 ms: sf::Shader::setUniformArray(std::__cxx11::basic_string<$> const&, fl... (1 times, avg 20 ms)
    19 ms: void std::vector<$>::_M_realloc_insert<$>(__gnu_cxx::__normal_iterat... (2 times, avg 9 ms)
    19 ms: sf::priv::SoundFileWriterFlac::check(std::__cxx11::basic_string<$> c... (1 times, avg 19 ms)
    19 ms: sf::Shader::setUniform(std::__cxx11::basic_string<$> const&, sf::Tex... (1 times, avg 19 ms)
    19 ms: __gnu_cxx::__normal_iterator<$> std::__find_if<$>(__gnu_cxx::__norma... (3 times, avg 6 ms)
    18 ms: std::unordered_map<std::__cxx11::basic_string<char, std::char_traits... (1 times, avg 18 ms)
    17 ms: sf::priv::SoundFileWriterOgg::open(std::__cxx11::basic_string<$> con... (1 times, avg 17 ms)
    16 ms: sf::View::View(sf::Rect<$> const&) (1 times, avg 16 ms)
    15 ms: void std::vector<$>::_M_realloc_insert<$>(__gnu_cxx::__normal_iterat... (1 times, avg 15 ms)
    15 ms: sf::priv::SoundFileWriterWav::open(std::__cxx11::basic_string<$> con... (1 times, avg 15 ms)
    15 ms: sf::Http::setHost(std::__cxx11::basic_string<$> const&, unsigned sho... (1 times, avg 15 ms)
    15 ms: sf::Shader::setUniform(std::__cxx11::basic_string<$> const&, sf::Vec... (3 times, avg 5 ms)
    13 ms: sf::Shader::setUniform(std::__cxx11::basic_string<$> const&, sf::Sha... (1 times, avg 13 ms)

*** Expensive headers:
28805 ms: C:/msys64/mingw64/x86_64-w64-mingw32/include/windows.h (included 25 times, avg 1152 ms), included via:
  WindowImpl.cpp.obj JoystickImpl.hpp JoystickImpl.hpp WindowsHeader.hpp  (1573 ms)
  CursorImpl.cpp.obj WindowsHeader.hpp  (1396 ms)
  Socket.cpp.obj SocketImpl.hpp SocketImpl.hpp WindowsHeader.hpp  (1350 ms)
  WindowImplWin32.cpp.obj WindowImplWin32.hpp WindowsHeader.hpp  (1336 ms)
  JoystickImpl.cpp.obj JoystickImpl.hpp JoystickImpl.hpp WindowsHeader.hpp  (1248 ms)
  SocketImpl.cpp.obj SocketImpl.hpp WindowsHeader.hpp  (1242 ms)
  ...

23851 ms: C:/OHW/SFML/src/SFML/System/Win32/WindowsHeader.hpp (included 20 times, avg 1192 ms), included via:
  WindowImpl.cpp.obj JoystickImpl.hpp JoystickImpl.hpp  (1574 ms)
  CursorImpl.cpp.obj  (1396 ms)
  Socket.cpp.obj SocketImpl.hpp SocketImpl.hpp  (1351 ms)
  WindowImplWin32.cpp.obj WindowImplWin32.hpp  (1337 ms)
  JoystickImpl.cpp.obj JoystickImpl.hpp JoystickImpl.hpp  (1249 ms)
  SocketImpl.cpp.obj SocketImpl.hpp  (1242 ms)
  ...

20790 ms: C:/OHW/SFML/include/SFML/System/Err.hpp (included 59 times, avg 352 ms), included via:
  TCP.cpp.obj Network.hpp System.hpp  (761 ms)
  Context.cpp.obj  (716 ms)
  SoundFileReaderWav.cpp.obj  (612 ms)
  EGLCheck.cpp.obj  (605 ms)
  Tennis.cpp.obj Graphics.hpp Window.hpp System.hpp  (598 ms)
  SoundFileReaderOgg.cpp.obj  (583 ms)
  ...

15978 ms: C:/msys64/mingw64/include/c++/11.2.0/bits/ios_base.h (included 80 times, avg 199 ms), included via:
  JoystickImpl.hpp Joystick.hpp String.hpp Utf.hpp iterator streambuf_iterator.h streambuf  (506 ms)
  String.hpp Utf.hpp iterator streambuf_iterator.h streambuf  (438 ms)
  String.hpp Utf.hpp iterator streambuf_iterator.h streambuf  (369 ms)
  Graphics.hpp Window.hpp System.hpp Err.hpp ostream ios  (357 ms)
  Network.hpp System.hpp Err.hpp ostream ios  (354 ms)
  iostream ostream ios  (348 ms)
  ...

13743 ms: C:/msys64/mingw64/include/c++/11.2.0/bits/basic_string.h (included 89 times, avg 154 ms), included via:
  IpAddress.hpp string  (279 ms)
  RenderWindow.hpp Image.hpp string  (275 ms)
  Text.hpp Font.hpp Texture.hpp Image.hpp string  (230 ms)
  JoystickImpl.hpp Joystick.hpp String.hpp Utf.hpp iterator streambuf_iterator.h streambuf ios_base.h locale_classes.h string  (226 ms)
  Shader.hpp string  (220 ms)
  Shader.hpp string  (217 ms)
  ...

12190 ms: C:/OHW/SFML/include/SFML/System/String.hpp (included 31 times, avg 393 ms), included via:
  JoystickImpl.cpp.obj JoystickImpl.hpp Joystick.hpp  (938 ms)
  JoystickManager.cpp.obj JoystickManager.hpp Joystick.hpp  (812 ms)
  WindowBase.cpp.obj WindowImpl.hpp Event.hpp Joystick.hpp  (795 ms)
  WindowImpl.cpp.obj WindowImpl.hpp Event.hpp Joystick.hpp  (736 ms)
  Clipboard.cpp.obj  (734 ms)
  String.cpp.obj  (726 ms)
  ...

10034 ms: C:/OHW/SFML/src/SFML/Network/Win32/SocketImpl.hpp (included 8 times, avg 1254 ms), included via:
  SocketImpl.cpp.obj  (1452 ms)
  Socket.cpp.obj SocketImpl.hpp  (1395 ms)
  IpAddress.cpp.obj SocketImpl.hpp  (1286 ms)
  TcpListener.cpp.obj SocketImpl.hpp  (1265 ms)
  Packet.cpp.obj SocketImpl.hpp  (1191 ms)
  UdpSocket.cpp.obj SocketImpl.hpp  (1159 ms)
  ...

9473 ms: C:/msys64/mingw64/include/c++/11.2.0/bits/stl_algobase.h (included 104 times, avg 91 ms), included via:
  RenderTexture.hpp Texture.hpp Image.hpp string char_traits.h  (204 ms)
  SoundFileReaderFlac.hpp vector  (186 ms)
  Packet.hpp string char_traits.h  (178 ms)
  Shader.hpp string char_traits.h  (171 ms)
  ImageLoader.hpp string char_traits.h  (170 ms)
  Shader.hpp string char_traits.h  (165 ms)
  ...

8626 ms: C:/OHW/SFML/include/SFML/Graphics.hpp (included 7 times, avg 1232 ms), included via:
  Tennis.cpp.obj  (1440 ms)
  OpenGL.cpp.obj  (1269 ms)
  Joystick.cpp.obj  (1223 ms)
  Win32.cpp.obj  (1211 ms)
  Shader.cpp.obj Effect.hpp  (1181 ms)
  Island.cpp.obj  (1176 ms)
  ...

8613 ms: C:/OHW/SFML/src/SFML/Network/SocketImpl.hpp (included 7 times, avg 1230 ms), included via:
  Socket.cpp.obj  (1395 ms)
  IpAddress.cpp.obj  (1287 ms)
  TcpListener.cpp.obj  (1265 ms)
  Packet.cpp.obj  (1191 ms)
  UdpSocket.cpp.obj  (1189 ms)
  SocketSelector.cpp.obj  (1151 ms)
  ...

  done in 0.0s.

This PR would make #1898 obsolete, but the work that has been done there is still greatly appreciated and the improvements brought by #1918 are still valid for doctest. Thanks!

@vittorioromeo vittorioromeo added this to the 3.0 milestone Dec 21, 2021
@Bromeon
Copy link
Member

Bromeon commented Dec 21, 2021 •
edited

Thanks for trying this out.

Do you know of any other trade-offs (pros + cons) of doctest vs. Catch2?
I've seen the FAQ on their GitHub page, but do you (or anybody) have personal experience with doctest?

While SFML compile times are better with doctest, they're nowhere near the "20 times" or "many times" advertised in the FAQ -- which in turn makes me skeptical how accurate their other promises are.

@vittorioromeo
Copy link
Member Author

I have personally used doctest both at work and in some personal projects (currently private repositories). I've never needed any advanced feature, but it does the job well for all my testing needs. I have used Catch in the past but after switching to doctest I have no reason to go back, it is just faster.

While compile times are better in SFML, it's nowhere near the "20 times" or "many times" advertised in the FAQ -- which in turn makes me skeptical how accurate their other promises are.

Their claims are not bogus. Check the analyses that I posted in the OP -- Catch dominates a huge chunk of the first compilation process analysis, and doctest doesn't even appear in the second one! Note that my analyses and timings are for a full complete build of SFML, including examples and tests. The benchmarks on doctest are way more specialized.

@binary1248
Copy link
Member

I remember trying out doctest the first time I heard of it at one of the author's presentations. I too noticed a perceivable build time improvement, especially when compared to Catch 1 (which was known for having bad build times back then). Although the library promotes writing the tests right next to the code I never really got used to that idea. I am not against replacing Catch2 with doctest if it offers many benefits to us given the current way our unit tests are written.

@eXpl0it3r eXpl0it3r added this to Backlog in SFML 3.0.0 via automation Dec 21, 2021
@vittorioromeo
Copy link
Member Author

I remember trying out doctest the first time I heard of it at one of the author's presentations. I too noticed a perceivable build time improvement, especially when compared to Catch 1 (which was known for having bad build times back then). Although the library promotes writing the tests right next to the code I never really got used to that idea. I am not against replacing Catch2 with doctest if it offers many benefits to us given the current way our unit tests are written.

I am not suggesting changing the way we write unit tests, I don't like putting them next to the code either. The benefits of this PR are strictly about improving compilation times and reducing CI load -- I think the benchmarks are quite impressive.

@vittorioromeo vittorioromeo mentioned this pull request Dec 21, 2021
Closed
@vittorioromeo
Copy link
Member Author

Note that doctest doesn't need the workaround for MinGW introduced in #1898, as well.

@ChrisThrasher
Copy link
Member

ChrisThrasher commented Dec 21, 2021 •
edited

How much of a bottleneck is building tests? I'm slowly getting up to speed with how CI is being handled with SFML but it appears that by far the biggest bottleneck to PRs are the non-GitHub CI jobs that seemingly take 10x longer than the GitHub jobs. Does this project only get so many minutes of CI time per month that we're trying to optimize for? Does this PR meaningfully speed up those other jobs that currently take so long to complete?

@vittorioromeo
Copy link
Member Author

How much of a bottleneck is building tests? I'm slowly getting up to speed with how CI is being handled with SFML but it appears that by far the biggest bottleneck to PRs are the non-GitHub CI jobs that seemingly take 10x longer than the GitHub jobs. Does this project only get so many minutes of CI time per month that we're trying to optimize for? Does this PR meaningfully speed up those other jobs that currently take so long to complete?

I have not measured the time it takes to perform a full CI run prior and after this PR. However, on my local machine a full rebuild of SFML is 4s faster after this PR. There are 47 CI checks. If we save a similar amount of time on a CI machine, we could be looking at around 188s of total time save. (@binary1248: any way we could measure that?)

Regardless, I would ask a different question: if both Catch2 and doctest provide the functionality that we require, but one is clearly faster to compile than the other, and transitioning from one to the other is trivial... why would we ever choose to pick the slower one?

@vittorioromeo vittorioromeo force-pushed the feature/replace_catch_with_doctest branch from d13d6bd to 6726724 Compare December 22, 2021 01:31
@vittorioromeo vittorioromeo mentioned this pull request Dec 23, 2021
Closed
@vittorioromeo vittorioromeo force-pushed the feature/replace_catch_with_doctest branch from 6726724 to d418e8d Compare December 23, 2021 20:13
@vittorioromeo
Copy link
Member Author

@eXpl0it3r, @Bromeon: could you approve this before we start making more contributions to the testing part of SFML (e.g. #1933 )?

@vittorioromeo vittorioromeo force-pushed the feature/replace_catch_with_doctest branch from d418e8d to 47a8af7 Compare December 24, 2021 02:28
@eXpl0it3r
Copy link
Member

@ChrisThrasher are you okay if we merge this soon and have your PRs having to be adapted to doctest?

Bromeon
Bromeon approved these changes Dec 24, 2021
View reviewed changes
Copy link
Member

@Bromeon Bromeon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm OK with this change.

Are there more objections, in favor of Catch2? @ChrisThrasher @eXpl0it3r or so?

eXpl0it3r
eXpl0it3r requested changes Dec 24, 2021
View reviewed changes
Copy link
Member

@eXpl0it3r eXpl0it3r left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be removed again, unless MinGW struggles as well with Doctest

SFML/test/CMakeLists.txt

Lines 16 to 19 in cd1cc62

# Work around MinGW struggling to process a file as large as catch.hpp
if(MINGW)
target_compile_options(sfml-test-main PUBLIC -Wa,-mbig-obj)
endif()

@Bromeon
Copy link
Member

Bromeon commented Dec 24, 2021 •
edited

What we should keep in mind: the future of Doctest is not very clear at this point -- doctest/doctest#554
Even then, at present it looks like a robust library. I hope that they find someone to maintain the project 🚀

@ChrisThrasher
Copy link
Member

are you okay if we merge this soon and have your PRs having to be adapted to doctest?

I'm fine adapting my MRs. Looks like a simple find-replace to make the switch.

What we should keep in mind: the future of Doctest is not very clear at this point

I'm not excited about the prospect of adopting a library without a clear maintenance plan just to save a few seconds of compile time, but I won't object to this.

@vittorioromeo vittorioromeo force-pushed the feature/replace_catch_with_doctest branch from 47a8af7 to 04099bd Compare December 24, 2021 13:31
@vittorioromeo
Copy link
Member Author

This should be removed again, unless MinGW struggles as well with Doctest

SFML/test/CMakeLists.txt

Lines 16 to 19 in cd1cc62

# Work around MinGW struggling to process a file as large as catch.hpp
if(MINGW)
target_compile_options(sfml-test-main PUBLIC -Wa,-mbig-obj)
endif()

Done. I remember it didn't need big-obj, but let's see if CI agrees.

What we should keep in mind: the future of Doctest is not very clear at this point -- doctest/doctest#554 Even then, at present it looks like a robust library. I hope that they find someone to maintain the project 🚀

Ditto... reverting to Catch is always an option if we ever need to, it would be easy to do.

@eXpl0it3r eXpl0it3r merged commit 29983aa into master Dec 24, 2021
SFML 3.0.0 automation moved this from Backlog to Done Dec 24, 2021
@eXpl0it3r eXpl0it3r deleted the feature/replace_catch_with_doctest branch December 24, 2021 14:06
@onqtam
Copy link

onqtam commented Dec 29, 2021

Pretty cool to see these results! Indeed the benchmarks in doctest are synthetic - they measure the compile time of simple translation units including only the framework header and nothing from the stdlib. I'm quite hopeful that the framework has a future!

@ChrisThrasher ChrisThrasher mentioned this pull request Jan 11, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Bromeon Bromeon Bromeon approved these changes

@eXpl0it3r eXpl0it3r eXpl0it3r approved these changes

@binary1248 binary1248 Awaiting requested review from binary1248

Assignees
No one assigned
Labels
feature m:unittest
Projects
SFML 3.0.0
Status: Done
SFML 3.0.0
  
Done
Milestone
3.0
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@vittorioromeo @Bromeon @binary1248 @ChrisThrasher @eXpl0it3r @onqtam