libhb/gtk: do not modify stderr struct directly on mingw

[marcosfrm]

Breaks mingw build when using UCRT:

../libhb/hb.c: In function 'redirect_thread_func': ../libhb/hb.c:2314:11: error: 'FILE' {aka 'struct _iobuf'} has no member named '_file'
 2314 |     stderr->_file = pfd[1];
      |           ^~

See:

mingw-w64/mingw-w64@fe226dd

Implement the same approach used by the GTK UI code:

HandBrake/gtk/src/application.c

Lines 307 to 311 in 3a9e37d

	#if defined(_WIN32)
	_dup2(pfd[1], _fileno(stderr));
	#else
	dup2(pfd[1], STDERR_FILENO);
	#endif

Tested on:

• Windows 10+ (via MinGW)
• macOS 10.13+
• Ubuntu Linux
• Fedora 39 mingw cross-compile (with both ucrt and msvcrt)

[bradleysepos]

Thanks for this. Where are you getting your mingw-w64 toolchain from, Fedora or the included build script, somewhere else?

[marcosfrm]

Official Fedora packages: mingw64-gcc-c++ (prefix x86_64-w64-mingw32) and ucrt64-gcc-c++ (prefix x86_64-w64-mingw32ucrt) are enough. Required dependencies are installed automatically.

https://docs.fedoraproject.org/en-US/packaging-guidelines/MinGW/

[sr55]

I'm afraid this doesn't work. We get no output on this in Windows.

[marcosfrm]

Sorry for not testing it. I was only compiling the CLI version and assumed it was working being there in the GTK code.

Found this: https://stackoverflow.com/a/25927081

This

    freopen("NUL", "w", stderr);
    _dup2(pfd[1], _fileno(stderr));

works on both ucrt and msvcrt here. @sr55 please give it a try.

[marcosfrm]

(I did not test the GTK part)

[jstebbins]

The mingw port of the gtk gui rarely gets tested as far as I know. I did the port just for kicks and then pretty much set it aside. It's probably full of bit-rot.

[marcosfrm]

With UCRT, all logging before the redirection is lost ("Compile-time hardening features are enabled" for example). It does not occur with MSVCRT somehow.

[galad87]

Looks good, tested on Windows and it looks fine.

[jstebbins]

Question, does this leave a file named "NUL" somewhere that needs to be cleaned up?

[marcosfrm]

It should not. NUL is the old DOS device which Windows will convert behind the scenes to \Device\Null, its /dev/null equivalent.

[jstebbins]

LGTM

But since this affects Windows, I'll let @sr55 make the final decision to merge

[sr55]

Sorry for the delayed response.

Looks like this is still causing a regression on Windows. We are losing some (not all) log messages now, rather than them being completely missing. As if a buffer is not being flushed correctly.

Before then After

Note, the After on the right is missing a bunch of logging that only later appears once the encode finalises.

[marcosfrm]

Let's try _IONBF, set in the GTK code already.

[marcosfrm]

Note: setvbuf(stderr, NULL, _IONBF, 0); called on non-Windows too.

[sr55]

That appears to behave better.