Running too many transcodings with x265 lead to a segfault in libx265

[daytimothy]

Problem Description

Running too many transcodings with x265 lead to a segfault in libx265.
Segfaults occur in RHEL and Ubuntu.
Occurs with bot AMD and Intel.

Activity Log, Crash Log or any other details

HandBrakeCLI[40880]: segfault at 0 ip 00007f41df2df014 sp 00007f3de9b53ec0 error 4 in libx265.so.199[7f41df288000+e98000]

What Operating System are you running?

Ubuntu 22.04, RHEL 9.1

What version of HandBrake are you running?

Latest release

Where did you download HandBrake from?

github

[sr55]

Can you please provide a log of the encode that failed,

[Nomis101]

Do you use the --hist-scenecut option in x265?

[daytimothy]

I will look for one but must check to make sure there is no company info in it.

[daytimothy]

Is that selected by default? I based my custom encodings based on the presets and don't remember enabling that setting.

[daytimothy]

Hopefully the files transferred.
command-H.265_MKV_AVX512_1080p30_10b-4.log
command-H.265_MKV_AVX512_1080p30_12b-2.log

[sr55]

Your running a very old, unofficial build of Handbrake that's not supported.

Please update to 1.5.1 or a nightly snapshot.

Note, Avoid using versions that come from Linux Repositories as they are often broken 3rd party builds.

[daytimothy]

For Ubuntu, I use the version in the repo.
For RHEL, I use version 1.5.1.
They both fail the same way.

[galad87]

Could you try a snapshot too? https://github.com/HandBrake/HandBrake-snapshots

[daytimothy]

There are 3 different flatpacks and the source code is from 2021.
What am I supposed to use?
Does this have a fix in it for this issue?

[galad87]

https://github.com/HandBrake/handbrake-snapshots/releases/tag/lin the first is the GUI, the third is the cli.

Probably not, it could be an issue of the x265 library, but the snapshots contain a more recent version, so it's worth a try.

[daytimothy]

I think it is a libx265 issue but I needed a jumping off point.
There are 3 different flatpacks and the source code is from 2021.
What am I supposed to use?

[galad87]

Where do you see that the source code is from 2021?
One flatpak is the GUI, another one is the CLI and one is a plugin to use the hardware QSV encoders. Get the one you prefer.

[sr55]

It'll be the automated "source" tars github creates.

Ignore them, They are not relevant and don't actually include HB source code. Sadly we can't remove them. The download links above have the flatpak GUI and CLI as needed.

[daytimothy]

I have never really messed with flatpacks. I usually just compile the source and run from the compiled directory.

[sr55]

compiled from source is fine as an alternative to using flatpak.

If you can recreate with compiling from source, would be interesting to see the log from that run as well please.