deps.ffmpeg: Switch Windows builds to native build toolchain

[PatTheMav]

Description

Adds an entirely Windows-based toolchain for building FFmpeg and its dependencies and replaces the current Linux-based toolchain.

Motivation and Context

Adds the ffmpeg package to Windows build scripts (selected by providing -PackageName ffmpeg to the build script).

While building Windows binaries via MINGW64, it leads to several small issues:

• MINGW64 does not support C++11 threading natively, instead it is implemented via POSIX threads, which makes all code depend on them (and in our case requires linking POSIX threads statically)
• Some 3rd party projects only support POSIX threads when they are not built via an MSVC-based toolchain, so once again a non-native threading implementation is uses
• Debug information in program database format (PDB) is not supported by the toolchain entirely as clang/gcc always embed debug information and there is no MING64 variant of the program database server (the service that collects symbols during compilation and generates PDBs)
• It creates another abstraction between the compiled code and its target platform - MINGW64 will always lag behind the actual feature level of the Windows platform

This PR builds all(!) current FFmpeg dependencies and FFmpeg itself natively on Windows with the MSVC toolchain (using either cl or clang-cl as compiler):

• zlib: Built natively
• libpng: Built natively
• opus: Built natively
• ogg: Built natively
• vorbis: Built natively
• libvpx: Uses shell support and make from MSYS2, compiled with MSBuild
• SVT-AV1: Built natively
• aom: Built natively
• x264: Uses shell support and make from MSYS2, compiled with cl
• libtheora: Built natively
• lame: Built natively using old (but still working) nmake script
• mbedtls: Built natively, but due to its code architecture as static libraries only
• srt: Built natively, using C++11 threading and static mbedtls
• librist: Built natively, using meson in a VS DevShell and embedded mbedtls
• ffmpeg: Built natively with Win32 threading support, statically linking all 3rd party libraries except for srt, librist, and x264 which are available as shared libraries

Note: The most impacting change on obs-deps is the need to also link to mbedtls statically due to the reasons above.

The scripts are entirely PowerShell-based, build dependencies are installed via winget, except for meson and ninja which are installed via pip3 on GitHub Action runners.

POSIX tools (pkg-config, yasm, nasm, make, and others) are installed via pacman inside the MSYS2 environment automatically.

How Has This Been Tested?

Tested on CI and locally on Windows 11.

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[PatTheMav]

Note: This PR does not add PDB creation - while some packages will automatically create PDBs because they are CMake-based (and thus use the MSVC toolchain natively), some do not (namely X264, libvpx, and FFmpeg itself), and others are built using clang-cl which does not generate PDBs (AOM).

It does however create a better starting point for adding Windows PDBs (see also #183 which does that for macOS).

[LeonarddeR]

I've been waiting for lame support in OBS to stream to an Icecast server for a long, long time. I replaced the ffmpeg build in the current release with the dlls produced by the CI of this pr and am very happy to see that this finally works now. So really looking forward for this to arrive!

[PatTheMav]

Updated this and the other PRs to follow a single dependency trail:

1. deps.qt: Remove Qt5 and update Qt6 build scripts #188
2. deps.windows: Update dependencies for CMake 3.0 upgrade #174
3. Add creation of macOS dSYM debug symbols to symbolicate release builds #183
4. deps.ffmpeg: Switch Windows builds to native build toolchain #186

Following this order will update each part of obs-deps one by one, such as that almost all parts of it are updated in some way at the end of the process.

[RytoEX]

Could you extract deps.ffmpeg: Add LICENSE files for svt-av1 to its own PR? I'd recently noticed this was missing, and was going to address it, but since you've already done that here, we might as well just use that. As a side note, licenses/svt-av1/LICENSE-BSD2.md only applies to SVT-AV1 up to v0.8.7. SVT-AV1 v0.8.8 and up use licenses/svt-av1/LICENSE.md, see https://gitlab.com/AOMediaCodec/SVT-AV1/-/commit/3af8029440fce01af8c72864a5f7ff4b9e607f31 for more info.

[RytoEX]

Submitting a preliminary review before I forget to do so. Initial testing seemed promising. Will investigate more soon.

[RytoEX]

Good news

OBS builds in Debug and RelWithDebInfo.

These worked:

• Encoders
  • AOM
  • SVT-AV1
  • AMF
  • NVENC
  • QSV
  • x264
• Recording
  • Standard
    • MKV
    • MP4
    • Fragmented MP4
  • Custom FFmpeg Output
    • MKV + libx264 + libvorbis
    • webm + libvpx-vp9 + libopus
• Streaming
  • RTMP
  • RTMPS
  • WHIP
  • SRT (local network)
  • RIST (local network)
• Sources
  • Browser
    • Default
    • FishGL/Aquarium
  • Color
  • Display Capture
    • WGC
  • Game Capture
  • Image
    • Animated GIF
    • JPG
    • PNG
  • Image Slide Show
  • Media
    • FLV
    • MP4
    • SRT ingest on local network (OBS to OBS)
    • RIST ingest on local network (OBS to OBS)
    • webm with transparency
  • Text (FreeType 2)
  • Text (GDI+)
  • Video Capture Device
  • VLC
    • playlist of files
  • Window Capture
    • BitBlt
    • WGC
• Filters
  • Chroma Key
  • Color Correction
• webm stingers
• Scripting
  • Python
  • Lua

Everything in my typical tests for those features worked.

Bad News

HLS streaming does not work. FFmpeg was compiled without HTTPS because the schannel test failed. Specifically, the check_func_headers test fails with:

error C2065: 'InitializeSecurityContext': undeclared identifier

I've also confirmed that this happens when trying to build FFmpeg locally. This seems to be caused by -DWINAPI_FAMILY=WINAPI_FAMILY_APP in the resulting defines for the test.

[RytoEX]

After further investigation, WINAPI_FAMILY_APP is deprecated anyway. Setting WINAPI_FAMILY to WINAPI_FAMILY_PC_APP does not work. Setting it to WINAPI_FAMILY_DESKTOP_APP works fine. Also, not setting it causes it to default to WINAPI_FAMILY_DESKTOP_APP, which also works.

@PatTheMav 's recent change removes the define entirely, so it will default to WINAPI_FAMILY_DESKTOP_APP with current Win10/11 SDK headers. The CI log indicates that securechannel and HTTPS are now enabled. Will re-test HLS shortly.

[RytoEX]

Retested HLS with the new changes and HLS now works, which means all of my tests passed.

Worth noting that some files may have different names (libsrt.dll -> srt.dll) which we may have to address via the updater, but that should be doable.

[RytoEX]

For posterity, these files are renamed:

• bin/libsrt.dll -> bin/srt.dll
• lib/libsrt.lib -> lib/srt.lib
• lib/libmbedcrypto.lib -> lib/mbedcrypto.lib
• lib/libmbedtls.lib -> lib/mbedtls.lib
• lib/libmbedx509.lib -> lib/mbedx509.lib

These files have been removed:

• bin/libmbedcrypto.dll
• bin/libmbedtls.dll
• bin/libmbedx509.dll
• lib/zlibstatic.lib

I believe these changes are intentional.