Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ffmpeg: Use latest builder #12007

Closed
wants to merge 5 commits into from
Closed

ffmpeg: Use latest builder #12007

wants to merge 5 commits into from

Conversation

maflcko
Copy link
Contributor

@maflcko maflcko commented May 28, 2024

No description provided.

Copy link

maflcko has previously contributed to projects/ffmpeg. The previous PR was #11714

@maflcko
Copy link
Contributor Author

maflcko commented May 28, 2024

cc @michaelni can you take a look here, please? When I tried this outside of OSS-Fuzz, it was also failing with clang-18, so it seems an issue unrelated to OSS-Fuzz. Let me know if I should submit a ticket to https://trac.ffmpeg.org/ instead.

@maflcko
Copy link
Contributor Author

maflcko commented May 28, 2024

For reference, to reproduce the link error locally, I used ./configure --cc="clang-18" --cxx="clang++-18 -std=c++11" --ld="clang++-18 -std=c++11" --enable-ossfuzz && make clean && make.

Looks like disabling the asm works around it for now.

However, some build issues remain.

@michaelni
Copy link
Contributor

cc @michaelni can you take a look here, please? When I tried this outside of OSS-Fuzz, it was also failing with clang-18, so it seems an issue unrelated to OSS-Fuzz. Let me know if I should submit a ticket to https://trac.ffmpeg.org/ instead.

My box here has no clang-18 nor is there a clang-18 package maybe you can post the errors you receive and what you did?

@maflcko
Copy link
Contributor Author

maflcko commented May 29, 2024

So far there are two related (?) build issues.

clang-16 (and 17, 18) with --enable-ossfuzz

To reproduce:

./configure --cc="clang-16" --cxx="clang++-16 -std=c++11" --ld="clang++-16 -std=c++11" --enable-ossfuzz && make clean && make

Output (taken from https://github.com/google/oss-fuzz/actions/runs/9270497789/job/25503641673#step:7:16578)

libavfilter/libavfilter.a(avf_showcqt_init.o): in function `ff_showcqt_init_x86':
avf_showcqt_init.c:(.text.unlikely.ff_showcqt_init_x86[ff_showcqt_init_x86]+0x98): undefined reference to `ff_showcqt_cqt_calc_sse'
/usr/bin/ld: avf_showcqt_init.c:(.text.unlikely.ff_showcqt_init_x86[ff_showcqt_init_x86]+0x141): undefined reference to `ff_showcqt_cqt_calc_sse3'
/usr/bin/ld: avf_showcqt_init.c:(.text.unlikely.ff_showcqt_init_x86[ff_showcqt_init_x86]+0x214): undefined reference to `ff_showcqt_cqt_calc_fma4'
/usr/bin/ld: avf_showcqt_init.c:(.text.unlikely.ff_showcqt_init_x86[ff_showcqt_init_x86]+0x2c0): undefined reference to `ff_showcqt_cqt_calc_avx'
/usr/bin/ld: avf_showcqt_init.c:(.text.unlikely.ff_showcqt_init_x86[ff_showcqt_init_x86]+0x380): undefined reference to `ff_showcqt_cqt_calc_fma3'

Workaround:

Add --disable-inline-asm --disable-asm.

libc++ build fails to link the fuzzer

The current OSS-Fuzz build still fails in this pull request (https://github.com/google/oss-fuzz/actions/runs/9276126037/job/25522451321?pr=12007#step:7:24150):

LD	tools/target_dem_aa_fuzzer
/usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 0x25
libavutil/libavutil.a(cpu.o): in function `ff_get_cpu_flags_x86':
cpu.c:(.text.ff_get_cpu_flags_x86[ff_get_cpu_flags_x86]+0x1dd): undefined reference to `ff_cpu_cpuid'
/usr/bin/ld: cpu.c:(.text.ff_get_cpu_flags_x86[ff_get_cpu_flags_x86]+0x249): undefined reference to `ff_cpu_cpuid'
/usr/bin/ld: cpu.c:(.text.ff_get_cpu_flags_x86[ff_get_cpu_flags_x86]+0x4f4): undefined reference to `ff_cpu_xgetbv'
/usr/bin/ld: cpu.c:(.text.ff_get_cpu_flags_x86[ff_get_cpu_flags_x86]+0x5e5): undefined reference to `ff_cpu_cpuid'
/usr/bin/ld: cpu.c:(.text.ff_get_cpu_flags_x86[ff_get_cpu_flags_x86]+0x863): undefined reference to `ff_cpu_cpuid'
/usr/bin/ld: cpu.c:(.text.ff_get_cpu_flags_x86[ff_get_cpu_flags_x86]+0x8b5): undefined reference to `ff_cpu_cpuid'
/usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 0x25
libavutil/libavutil.a(imgutils_init.o): in function `ff_image_copy_plane_uc_from_x86':
imgutils_init.c:(.text.ff_image_copy_plane_uc_from_x86[ff_image_copy_plane_uc_from_x86]+0xc6): undefined reference to `ff_image_copy_plane_uc_from_sse4'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [Makefile:62: tools/target_dem_aa_fuzzer] Error 1

I tried without libc++ outside of the OSS-Fuzz container and it worked, so as a next step one would have to debug inside the OSS-Fuzz container or recreate a libc++ build outside of it (which is tedious, as it requires building all ffmpeg dependencies with libc++).

@michaelni
Copy link
Contributor

libavfilter/libavfilter.a(avf_showcqt_init.o): in function ff_showcqt_init_x86': avf_showcqt_init.c:(.text.unlikely.ff_showcqt_init_x86[ff_showcqt_init_x86]+0x98): undefined reference to ff_showcqt_cqt_calc_sse'
Workaround:

Add --disable-inline-asm --disable-asm.

you should disable asm IF thats what you intend. The fuzzer already should test with and without asm by switching at runtime. So disabling seems not correct.

The undefined reference error (and maybe some other errors too) seems to come from a wrong extern_prefix set by configure.
I have not investigated this beyond that but the errors disappear if its commented out.

I dont need clang-16-18 with ossfuzz :)
If you do, then please investigate and submit a clean patch to ffmpeg-devel. I assume some minor fix around the extern_prefix code could resolve this

thx

@maflcko
Copy link
Contributor Author

maflcko commented Jun 19, 2024

I dont need clang-16-18 with ossfuzz :)

The motivation for this change is that for an OSS-Fuzz infra upgrade, all projects should be compiled with at least clang-18.

I took another look and it seems that the first error happens after clang enabled -fsanitize-address-use-odr-indicator by default. So the first build error is reproducible with any clang version, if -fsanitize-address-use-odr-indicator is set. And conversely, it does not happen when -fno-sanitize-address-use-odr-indicator is set.

Though, that still doesn't fix the second error.

@maflcko
Copy link
Contributor Author

maflcko commented Jul 15, 2024

Closing in favour of #12211

@maflcko maflcko closed this Jul 15, 2024
@maflcko maflcko deleted the patch-2 branch July 15, 2024 07:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants