Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bazel support #83

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Bazel support #83

wants to merge 11 commits into from

Conversation

zaucy
Copy link

@zaucy zaucy commented May 16, 2022

I'm using bazel to build my projects and I was hoping to use c4core and rapidyaml. I figured I'd share my bazel build files upstream. If this gets merged then including this repository in other bazel based projects becomes very trivial.

If this gets approved I was hoping to also contribute the files necessary to add c4core and rapid yaml to the bazel central registry.

The test library doctest already had bazel support so I added the tests in as well! If you want I could also add some bazel-based CI jobs.

If this PR gets a positive reception I'll make another one in the rapidyaml repository as well.

Let me know what you think!

@codecov
Copy link

codecov bot commented May 23, 2022

Codecov Report

Merging #83 (e3d520d) into master (b8f2110) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master      #83   +/-   ##
=======================================
  Coverage   94.63%   94.63%           
=======================================
  Files          54       54           
  Lines       12524    12524           
=======================================
  Hits        11852    11852           
  Misses        672      672           
Impacted Files Coverage Δ
test/c4/test.hpp 94.64% <ø> (ø)
test/test_charconv.cpp 99.94% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@biojppm
Copy link
Owner

biojppm commented May 23, 2022

Thanks for the PR, and sorry for the delay -- github's notifications are really broken for me.

I am not familiar with Bazel, but AFAICT the structure is OK, and I see that you took good care to deal with the build options.

To merge this, adding a CI job would be needed, all the more because it hardcodes the lists of source files and tests, which would otherwise diverge from the main cmake project.

Another thing that will be needed is adding notes to the main README.

WORKSPACE Outdated Show resolved Hide resolved
BUILD.bazel Show resolved Hide resolved
@biojppm
Copy link
Owner

biojppm commented Aug 20, 2022

@zaucy do you intend to proceed with this PR?

@zaucy
Copy link
Author

zaucy commented Aug 20, 2022

Yes I do! Thanks for the reminder

@zaucy
Copy link
Author

zaucy commented Aug 21, 2022

I added some bazel specific CI. The charconv test is failing and I'm unsure why.

Test failure output
Executing tests from //test:charconv
-----------------------------------------------------------------------------
[doctest] doctest version is "2.4.8"
[doctest] run with "--help" for options
===============================================================================
test/test_charconv.cpp(2141):
TEST CASE:  atof.basic<float>

test/test_charconv.cpp(2176): ERROR: CHECK_EQ( memcmp(&rval, &nan, sizeof(T)), 0 ) is NOT correct!
  values: CHECK_EQ( 1, 0 )

===============================================================================
test/test_charconv.cpp(2141):
TEST CASE:  atof.basic<double>

test/test_charconv.cpp(2176): ERROR: CHECK_EQ( memcmp(&rval, &nan, sizeof(T)), 0 ) is NOT correct!
  values: CHECK_EQ( 1, 0 )

===============================================================================
[doctest] test cases:     303 |     301 passed | 2 failed | 0 skipped
[doctest] assertions: 1305529 | 1305527 passed | 2 failed |
[doctest] Status: FAILURE!

@biojppm
Copy link
Owner

biojppm commented Aug 22, 2022

The test failure is curious. It occurs on the last line of this test, so what's happening here is that atof("nan", &v) does not result in std::numeric_limits<float>::quiet_NaN() (same for double).

But this works ok in every tested architecture/compiler in the unit tests (just look at the CI runs), including exotics as s390. I don't see how this could be related to this PR, and suspect instead that you caught something in your run setup. But I may be wrong.

What is your cpu/platform/compiler?

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
BUILD.bazel Show resolved Hide resolved
@biojppm
Copy link
Owner

biojppm commented Aug 22, 2022

@zaucy thanks for all the hard work, your PR is very thorough. The remarks are all minor, and the main sticking point is to find out whether the problem with the failed test has anything to do with this changeset or with the bazel environment, which I suspect is not the case.

@zaucy
Copy link
Author

zaucy commented Aug 22, 2022

I was mainly testing on Windows x86_64 using msvc cl. It also failed on my branches github actions for the windows-latest agent https://github.com/zaucy/c4core/runs/7936292285?check_suite_focus=true

I just ran the tests on a Linux machine and realized there are a few things that need to be configured for the tests still. I'll update my PR later today.

@biojppm
Copy link
Owner

biojppm commented Aug 22, 2022

ok, so given that all CI runs other than bazel are ok, this leads us to conclude that bazel, or the way it runs the tests, has something to do with the failure. Would you agree?

If so, do you have an idea what may be happening here?

@zaucy
Copy link
Author

zaucy commented Aug 22, 2022

It must be. I can't see why else. I'll investigate further later today.

@biojppm
Copy link
Owner

biojppm commented Aug 22, 2022

It also occurred to me that it may have something to do with how bazel compiles the code. The fact that it happens with std::numeric_limits<>::quiet_NaN() and nowhere else also suggests this. Looking at the documentation, std::numeric_limits<T>::has_quiet_NaN must be true for quiet_NaN() to be meaningful. Does has_quiet_NaN differ between bazel and cmake? Maybe bazel is adding a compile flag which changes this behavior?

@zaucy
Copy link
Author

zaucy commented Aug 23, 2022

Does has_quiet_NaN differ between bazel and cmake? Maybe bazel is adding a compile flag which changes this behavior?

I'm unaware of any differences in regards to that. I've extracted the compile/link commands that were generated by bazel. Does anything look like it would cause quiet NaN to differ?

Compile commands
cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/memory_resource.obj /c src/c4/memory_resource.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/base64.obj /c src/c4/base64.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/char_traits.obj /c src/c4/char_traits.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Iexternal/doctest /Ibazel-out/x64_windows-fastbuild/bin/external/doctest /Itest /Ibazel-out/x64_windows-fastbuild/bin/test /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/test/_objs/lib/main.obj /c test/c4/main.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/error.obj /c src/c4/error.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Iexternal/doctest /Ibazel-out/x64_windows-fastbuild/bin/external/doctest /Iexternal/bazel_tools /Ibazel-out/x64_windows-fastbuild/bin/external/bazel_tools /Itest /Ibazel-out/x64_windows-fastbuild/bin/test /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/test/_objs/charconv/test_charconv.obj /c test/test_charconv.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Iexternal/doctest /Ibazel-out/x64_windows-fastbuild/bin/external/doctest /Itest /Ibazel-out/x64_windows-fastbuild/bin/test /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/test/_objs/lib/test.obj /c test/c4/libtest/test.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/language.obj /c src/c4/language.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/utf.obj /c src/c4/utf.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/format.obj /c src/c4/format.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/_objs/c4core/memory_util.obj /c src/c4/memory_util.cpp

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /Iexternal/doctest /Ibazel-out/x64_windows-fastbuild/bin/external/doctest /DDOCTEST_CONFIG_IMPLEMENT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/external/doctest/doctest/_objs/custom_main/dummy-main.obj /c bazel-out/x64_windows-fastbuild/bin/external/doctest/doctest/dummy-main.cc

cl.exe /nologo /DCOMPILER_MSVC /DNOMINMAX /D_WIN32_WINNT=0x0601 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS /bigobj /Zm500 /EHsc /wd4351 /wd4291 /wd4250 /wd4996 /I. /Ibazel-out/x64_windows-fastbuild/bin /Iexternal/doctest /Ibazel-out/x64_windows-fastbuild/bin/external/doctest /Itest /Ibazel-out/x64_windows-fastbuild/bin/test /Isrc /Ibazel-out/x64_windows-fastbuild/bin/src /DC4CORE_NO_FAST_FLOAT /showIncludes /MD /Od /Z7 /wd4117 -D__DATE__="redacted" -D__TIMESTAMP__="redacted" -D__TIME__="redacted" /Fobazel-out/x64_windows-fastbuild/bin/test/_objs/lib/archetypes.obj /c test/c4/libtest/archetypes.cpp
Link Commands
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\link.exe /nologo /OUT:bazel-out/x64_windows-fastbuild/bin/test/charconv.exe bazel-out/x64_windows-fastbuild/bin/test/_objs/charconv/test_charconv.obj bazel-out/x64_windows-fastbuild/bin/test/lib.lib bazel-out/x64_windows-fastbuild/bin/external/doctest/doctest/custom_main.lib bazel-out/x64_windows-fastbuild/bin/c4core.lib /SUBSYSTEM:CONSOLE /MACHINE:X64 /DEFAULTLIB:msvcrt.lib /DEBUG:FASTLINK /INCREMENTAL:NO

C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\lib.exe /nologo /OUT:bazel-out/x64_windows-fastbuild/bin/c4core.lib /MACHINE:X64 bazel-out/x64_windows-fastbuild/bin/_objs/c4core/base64.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/char_traits.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/error.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/format.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/language.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/memory_resource.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/memory_util.obj bazel-out/x64_windows-fastbuild/bin/_objs/c4core/utf.obj /ignore:4221
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\lib.exe /nologo /OUT:bazel-out/x64_windows-fastbuild/bin/test/lib.lib /MACHINE:X64 bazel-out/x64_windows-fastbuild/bin/test/_objs/lib/archetypes.obj bazel-out/x64_windows-fastbuild/bin/test/_objs/lib/test.obj bazel-out/x64_windows-fastbuild/bin/test/_objs/lib/main.obj /ignore:4221
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.32.31326\bin\HostX64\x64\lib.exe /nologo /OUT:bazel-out/x64_windows-fastbuild/bin/external/doctest/doctest/custom_main.lib /MACHINE:X64 bazel-out/x64_windows-fastbuild/bin/external/doctest/doctest/_objs/custom_main/dummy-main.obj /ignore:4221

@biojppm
Copy link
Owner

biojppm commented Aug 24, 2022

I've looked through the compile flags and nothing there seems to warrant this behavior. One last idea/try on this front: in the failing test, add static_assert(std::numeric_limits<T>::has_quiet_NaN, "quiet indeed") and trigger the tests again to compare bazel with the rest. Will the assertion fire differently?

@biojppm
Copy link
Owner

biojppm commented Aug 25, 2022

@zaucy I have rebased and pushed the quiet nan test to your branch. Let's see what the results are.

@biojppm
Copy link
Owner

biojppm commented Aug 25, 2022

I'm having trouble building the test using bazel:

$ bazel build test/charconv
INFO: Analyzed target //test:charconv (27 packages loaded, 275 targets configured).
INFO: Found 1 target...
ERROR: /home/jpmag/proj/c4core/test/BUILD.bazel:76:8: Compiling test/test_charconv.cpp failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer '-std=c++0x' -MD -MF ... (remaining 33 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from src/c4/std/vector.hpp:10,
                 from src/c4/std/std.hpp:6,
                 from test/test_charconv.cpp:4:
src/c4/substr.hpp: In instantiation of 'OStream& c4::operator<<(OStream&, basic_substring<C>) [with OStream = doctest::detail::MessageBuilder; C = char]':
test/test_charconv.cpp:2107:9:   required from 'void c4::test_rtoa(substr, Real, int, const char*, const char*, const char*, const char*, const char*) [with Real = float; substr = basic_substring<char>]'
test/test_charconv.cpp:2143:14:   required from here
src/c4/substr.hpp:1865:8: error: 'struct doctest::detail::MessageBuilder' has no member named 'write'
 1865 |     os.write(s.str, s.len);
      |     ~~~^~~~~
src/c4/substr.hpp: In instantiation of 'OStream& c4::operator<<(OStream&, basic_substring<C>) [with OStream = doctest::detail::MessageBuilder; C = const char]':
test/test_charconv.cpp:2454:5:   required from 'void c4::to_chars_roundtrip(char (&)[N], csubstr) [with long unsigned int N = 128; csubstr = basic_substring<const char>]'
test/test_charconv.cpp:2527:23:   required from here
src/c4/substr.hpp:1865:8: error: 'struct doctest::detail::MessageBuilder' has no member named 'write'
Target //test:charconv failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 5.014s, Critical Path: 4.08s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

I noticed the bazel workspace specifies doctest 2.4.8, while cmake will pick 2.4.6. However, then I tried building in cmake with doctest 2.4.8 and it builds without any problem. Do you know what may be going on in here?

@zaucy
Copy link
Author

zaucy commented Aug 25, 2022

I saw that too. I haven't had a chance to investigate what the difference is. I'm not too familiar with doctest.

It doesn't happen on Windows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants