Skip to content
This repository has been archived by the owner on Jun 2, 2022. It is now read-only.

Unbreak build on FreeBSD #215

Merged
merged 1 commit into from
Jan 3, 2020
Merged

Unbreak build on FreeBSD #215

merged 1 commit into from
Jan 3, 2020

Conversation

jbeich
Copy link

@jbeich jbeich commented Dec 16, 2019

Tested on FreeBSD 11.3 (Clang/libc++ 8.0), 12.0 (Clang/libc++ 6.0), 13.0 (Clang/libc++ 9.0) and explicitly with GCC 8.1/9.1/7.4/6.5/5.5.

  • FreeBSD and OpenBSD use libc++ by default on x86-based and arm-based architectures
  • FreeBSD 11.4/12.2/13.0 no longer has libc++fs due to llvm/llvm-project@cc37af7a3631
  • libstdc++fs is only required on GCC 8 due to gcc-mirror/gcc@c6e37a9f5734 and gcc-mirror/gcc@3935f409704b
  • Explicitly linking against libc++/libstdc++ is not required if the target uses C++ files or has LINKER_LANGUAGE CXX property set
  • Since auto-detection is error-prone let downstream override FILE_SYSTEM_LIB
  • Fall back to <experimental/filesystem> on non-Linux or Clang/libc++ as well

@jbeich
CMake: unbreak build on distributions that use libc++
858fed6 
src/modelHandler_OpenCL.cpp:53:10: fatal error: 'filesystem' file not found
 #include <filesystem>
          ^~~~~~~~~~~~
ld: error: unable to find library -lstdc++fs
@kattjevfel
Copy link

kattjevfel commented Dec 16, 2019 •
edited
Loading

This breaks on linux (gcc 9.2.0)

waifu2x-converter-cpp/out [freebsd] » ./waifu2x-converter-cpp -i 11153579.png              
CUDA: GeForce GTX 1080
Processing file [1/1] "/home/katt/build/waifu2x-converter-cpp/out/11153579.png":
Scaling image from 460x460 to 920x920

Step 01/02: Denoising
Proccessing [1/1] slices
Processing block, column (01/01), row (01/01) ...
[1]    331617 segmentation fault (core dumped)  ./waifu2x-converter-cpp -i 11153579.png

gdb:

Reading symbols from /home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp...

warning: core file may not match specified executable file.
[New LWP 333262]
[New LWP 333268]
[New LWP 333275]
[New LWP 333266]
[New LWP 333273]
[New LWP 333267]
[New LWP 333269]
[New LWP 333272]
[New LWP 333270]
[New LWP 333277]
[New LWP 333271]
[New LWP 333265]
[New LWP 333276]
[New LWP 333274]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `./waifu2x-converter-cpp --log-level 2 -i 11153579'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007fcaf69c3d96 in Buffer::prealloc (this=0x56473955a8a0, conv=0x5647395671d0, env=0x564739551520) at /home/katt/build/waifu2x-converter-cpp/src/Buffer.cpp:336
336                                     cuCtxPushCurrent(dev->context);
[Current thread is 1 (Thread 0x7fcaf21ee400 (LWP 333262))]
/usr/lib/../share/gcc-9.2.0/python/libstdcxx/v6/xmethods.py:731: SyntaxWarning: list indices must be integers or slices, not str; perhaps you missed a comma?
  refcounts = ['_M_refcount']['_M_pi']

backtrace:

#0  0x00007fcaf69c3d96 in Buffer::prealloc (this=0x56473955a8a0, conv=0x5647395671d0, env=0x564739551520) at /home/katt/build/waifu2x-converter-cpp/src/Buffer.cpp:336
#1  0x00007fcaf69adaa4 in w2xc::convertWithModelsBlockSplit (conv=0x5647395671d0, env=0x564739551520, inputPlane_2=..., outputPlane_2=..., models=std::vector of length 7, capacity 8 = {...}, 
    flops=0x5647395671e8, blockSize=512, fmt=w2xc::IMAGE_RGB_F32, log_level=2) at /home/katt/build/waifu2x-converter-cpp/src/convertRoutine.cpp:381
#2  0x00007fcaf69ac824 in w2xc::convertWithModels (conv=0x5647395671d0, env=0x564739551520, inputPlane=..., outputPlane=..., models=std::vector of length 7, capacity 8 = {...}, 
    flops=0x5647395671e8, blockSize=0, fmt=w2xc::IMAGE_RGB_F32, log_level=2) at /home/katt/build/waifu2x-converter-cpp/src/convertRoutine.cpp:69
#3  0x00007fcaf69b231c in apply_denoise (conv=0x5647395671d0, image=..., denoise_level=1, blockSize=0, fmt=w2xc::IMAGE_RGB_F32) at /home/katt/build/waifu2x-converter-cpp/src/w2xconv.cpp:765
#4  0x00007fcaf69b4d5e in w2xconv_convert_mat (conv=0x5647395671d0, image_dst=0x7ffea4511d20, image_src=0x7ffea4511cc0, denoise_level=1, scale=2, blockSize=0, background=..., has_alpha=true, 
    dst_alpha=true) at /home/katt/build/waifu2x-converter-cpp/src/w2xconv.cpp:1882
#5  0x00007fcaf69b642e in w2xconv_convert_file (conv=0x5647395671d0, dst_path=0x56473955ce60 "/home/katt/build/waifu2x-converter-cpp/out/11153579_[L1][x2.00].png", 
    src_path=0x56473973a3a0 "/home/katt/build/waifu2x-converter-cpp/out/11153579", denoise_level=1, scale=2, blockSize=0, imwrite_params=0x7ffea4512170)
    at /home/katt/build/waifu2x-converter-cpp/src/w2xconv.cpp:2252
#6  0x0000564738eff155 in convert_file (info=..., inputName=filesystem::path "11153579", output=filesystem::path "auto") at /home/katt/build/waifu2x-converter-cpp/src/main.cpp:518
#7  0x0000564738f02abd in main (argc=5, argv=0x7ffea4513a48) at /home/katt/build/waifu2x-converter-cpp/src/main.cpp:1059

@jbeich
Copy link
Author

jbeich commented Dec 16, 2019 •
edited
Loading

@kattus, I can't reproduce with GCC 9.1.0 on FreeBSD using CPU or OpenCL. Can you check if CLI and the library are linked against libstdc++? Does the crash affect only CUDA?

$ waifu2x-converter-cpp --list-processor
   0: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz     (FMA       ): num_core=8
   1: Intel(R) HD Graphics Skylake Desktop GT2     (OpenCL    ): num_core=24

$ readelf -d waifu2x-converter-cpp | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libw2xc.so]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libthr.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]

$ readelf -d libw2xc.so | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libthr.so.3]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.3.4]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.5]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.7]

@kattjevfel
Copy link

Yeah I should've done that in my initial post, turns out it only crashes when using CUDA.
Here's the other stuff though..

waifu2x-converter-cpp/out [freebsd] » readelf -d waifu2x-converter-cpp | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libw2xc.so]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

waifu2x-converter-cpp/out [freebsd] » readelf -d libw2xc.so | fgrep NEEDED
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgcodecs.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libdl.so.2]
 0x0000000000000001 (NEEDED)             Shared library: [libpthread.so.0]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_imgproc.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libopencv_core.so.4.1]
 0x0000000000000001 (NEEDED)             Shared library: [libstdc++.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libm.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [libgcc_s.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]

@jbeich
Copy link
Author

jbeich commented Dec 16, 2019

@kattus, does the following patch help? CUDA runtime doesn't use libstdc++, and CUDA toolkit is not relevant after build. I'd expect a crash if libstdc++fs symbols satisfied references from old libstdc++, not the other way around.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index bc7cd67..18005a2 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -79,7 +79,7 @@ if(APPLE AND CMAKE_CXX_COMPILER_ID MATCHES "^(Apple)?Clang$")
 	endif()
 elseif(UNIX OR MINGW)
 # https://gitlab.kitware.com/cmake/cmake/issues/17834
-	if (NOT FILE_SYSTEM_LIB AND CMAKE_CXX_COMPILER_VERSION VERSION_LESS 9)
+	if (NOT FILE_SYSTEM_LIB)
 		set(FILE_SYSTEM_LIB "stdc++fs")
 	endif()
 elseif(MSVC)

@kattjevfel
Copy link

kattjevfel commented Dec 16, 2019 •
edited
Loading

It still crashes but with a different stacktrace and debug output:
stacktrace:

Stack trace of thread 344678:
#0  0x00007f05a14a68a8 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x768a8)
#1  0x00007f05a1493002 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x63002)
#2  0x00007f05a147108f n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x4108f)
#3  0x00007f05a1471428 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x41428)
#4  0x00007f05a148fb2e n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x5fb2e)
#5  0x00007f05a1490ee0 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x60ee0)
#6  0x00007f05a148f824 n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x5f824)
#7  0x00007f05a149531c n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x6531c)
#8  0x00007f05a1497d5e n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x67d5e)
#9  0x00007f05a149942e n/a (/home/katt/build/waifu2x-converter-cpp/out/libw2xc.so + 0x6942e)
#10 0x0000557dcf9e7155 n/a (/home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp + 0x13155)
#11 0x0000557dcf9eaabd n/a (/home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp + 0x16abd)
#12 0x00007f059e15f153 __libc_start_main (libc.so.6 + 0x27153)
#13 0x0000557dcf9e59de n/a (/home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp + 0x119de)

gdb:

Reading symbols from /home/katt/build/waifu2x-converter-cpp/out/waifu2x-converter-cpp...

warning: core file may not match specified executable file.
[New LWP 344678]
[New LWP 344687]
[New LWP 344691]
[New LWP 344684]
[New LWP 344685]
[New LWP 344683]
[New LWP 344689]
[New LWP 344682]
[New LWP 344688]
[New LWP 344686]
[New LWP 344690]
[New LWP 344681]
[New LWP 344693]
[New LWP 344692]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
Core was generated by `./waifu2x-converter-cpp -i 11153579'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f05a14a68a8 in Buffer::get_write_ptr_cuda (this=0x557dd13ed6c0, env=0x557dd13d3520, devid=0) at /home/katt/build/waifu2x-converter-cpp/src/Buffer.cpp:218
218         cuCtxPushCurrent(dev->context);
[Current thread is 1 (Thread 0x7f059ccd1400 (LWP 344678))]

I made sure I had set -DCMAKE_BUILD_TYPE='Debug' (re-compiled 3 times) so idk why it shows n/a

@kattjevfel
Copy link

Ignore everything I previously said, it turned I had CUDA uninstalled without me noticing, and even when compiled without CUDA support, it will try to use it. It has nothing to do with your PR. Very sorry for this.

@DeadSix27 DeadSix27 merged commit db532b1 into DeadSix27:master Jan 3, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@DeadSix27 DeadSix27 DeadSix27 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jbeich @kattjevfel @DeadSix27