build(win): use UCRT64 environment instead of MinGW64.

[tez011]

Description

#2149 was reverted in #2320 and needs to be split into three changes:

• C++20
• UCRT64 (this one!)
• WGC

If msys2/MINGW-packages#20477 is integrated, WGC can proceed without this change.

Screenshot

Issues Fixed or Closed

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected) 🤪
• Dependency update (updates to dependencies)
• Documentation update (changes to documentation)
• Repository update (changes to repository files, e.g. .github/...)

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have added or updated the in code docstring/documentation-blocks for new or existing methods/components

Branch Updates

LizardByte requires that branches be up-to-date before merging. This means that after any PR is merged, this branch
must be updated before it can be merged. You must also
Allow edits from maintainers.

• I want maintainers to keep my branch updated

[psyke83]

This resolves the regex CPU/memory hog issue:

diff --git a/src/main.cpp b/src/main.cpp
index a3901448..7672bac6 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -102,8 +102,8 @@ main(int argc, char *argv[]) {
   task_pool_util::TaskPool::task_id_t force_shutdown = nullptr;

 #ifdef _WIN32
-  // Switch default C standard library locale to UTF-8 on Windows 10 1803+
-  setlocale(LC_ALL, ".UTF-8");
+  // Switch default C standard library locale to UTF-16 on Windows 10 1803+
+  setlocale(LC_ALL, ".UTF-16");
 #endif

 #pragma GCC diagnostic push

Setting to UTF16 or UTF32 resolves the slowdown, but I'm not sure if that actually breaks our current regex parsing?

[psyke83]

Relevant discussion (see also in comments): https://stackoverflow.com/questions/59067280/does-stdwregex-support-utf-16-unicode-or-only-ucs-2

Setting to UTF16 doesn't break my en-IE keyboard mappings, but it could have consequences for other languages and may also break log parsing (see: cadd3da)

[psyke83]

I retested the #1477 by printing some Chinese characters with .UTF-16 set, but didn't change the UTF-8 conversion.

The result: Sunshine doesn't crash, regex doesn't consume CPU or memory, and the Chinese characters print in the web UI log correctly. This may be a viable fix, but @cgutman will probably want to review that change to make sure no other changes are required.

[psyke83]

Apologies for the spam, but my observations were flawed. It appears that MINGW64 builds were not setting the UTF-8 codepage correctly, so the default C codepage was in use. UCRT64 builds do set a UTF-8 codepage, but cause the std::regex issue. Leaving the codepage to C (not UTF-16, as that was falling back to C on UCRT64 as well) seems to be the correct solution.

More context: msys2/MINGW-packages#20477 (comment)

[psyke83]

Two points on MSVCRT in the MSYS2 environments documentation:

• It doesn't support the UTF-8 locale

This explains the behaviour we're seeing. I didn't mention in my prior comment, but setting the system default locale via std::setlocale(LC_ALL, ""); does apply correctly on both MINGW64 and UCRT64. For my system, it sets English_Ireland.1252 for both runtimes, and regex doesn't exhibit any issues with either runtime.

• Binaries linked with MSVCRT should not be mixed with UCRT ones because the internal structures and data types are different. (More strictly, object files or static libraries built for different targets shouldn't be mixed. DLLs built for different CRTs can be mixed as long as they don't share CRT objects, e.g. FILE*, across DLL boundaries.) Same rule is applied for MSVC compiled binaries because MSVC uses UCRT by default (if not changed).

This suggests to me that we need a complementary PR for build-deps to also switch to UCRT64, so that the statically linked ffmpeg libraries don't cause unforseen issues.

[psyke83]

Regarding the UAC failing to trigger via shortcut launch: std::filesystem::exists returns false due to the restrictive permissions of the credentials folder. This may be a bug in UCRT64 or perhaps it is the more correct behaviour; I'm not sure.

A workaround (or perhaps the only solution) is the following:

diff --git a/src_assets/windows/misc/migration/migrate-config.bat b/src_assets/windows/misc/migration/migrate-config.bat
index 65d00d8f..aa8d10aa 100644
--- a/src_assets/windows/misc/migration/migrate-config.bat
+++ b/src_assets/windows/misc/migration/migrate-config.bat
@@ -39,9 +39,10 @@ rem Create the credentials directory if it wasn't migrated or already existing
 if not exist "%NEW_DIR%\credentials\" mkdir "%NEW_DIR%\credentials"

 rem Disallow read access to the credentials directory for normal users
-rem Note: We must use the SID directly because "Administrators" is localized
+rem Note: We must use the SIDs directly because "Users" and "Administrators" are localized
 icacls "%NEW_DIR%\credentials" /inheritance:r
 icacls "%NEW_DIR%\credentials" /grant:r *S-1-5-32-544:(OI)(CI)(F)
+icacls "%NEW_DIR%\credentials" /grant:r *S-1-5-32-545:(R)

 rem Migrate the covers directory
 if exist "%OLD_DIR%\covers\" (

This will grant the Users aka (S-1-5-32-545) group read access for the folder (but not its contents). Technically it may reduce security because users can now list folder contents, but they still can't read or make any changes to the files contained within.

Edit: amended to use the SID to avoid localization issues.

[FrogTheFrog]

Whatever you end up doing, do not use std::setlocale(LC_ALL, "");.

On Windows you can actually set global locale to be UTF-8 and calling std::setlocale(LC_ALL, ""); will then set it to UTF-8 instead of the default C, and then we're back at the square 1.

[tez011]

oof, I'm embarrassed by that. Should have read setlocale docs before blindly setting it to what I thought was the 'default' locale. Thanks.

https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-170

At program startup, the equivalent of the following statement is executed:
setlocale( LC_ALL, "C" );

I guess it's possible we don't need to do this at all, but I think it might be a good idea to leave this in there anyway, so there isn't any appetite to add back UTF-8 in the future.

[FrogTheFrog]

msys2/MINGW-packages#20477 has been merged. Maybe we don't need to migrate to ucrt anymore (but still remove the UTF-8 locale)?

[ReenigneArcher]

It seems that mingw is becoming legacy, so probably worth migrating anyway. The WGC just won't depend on this now.

[ReenigneArcher]

@tez011 did you see @psyke83 's PR into your branch? tez011#1

[tez011]

Thanks for the ping and for patience. I didn't get any email notifications about that pull request until yours... though I did see it yesterday and thought it was closed.

I guess if the service launches sunshine as administrator, it'll have access to the configuration files specified and the patch will work good. I've merged that change. Thanks for the patch @psyke83 !

[psyke83]

Thanks for the ping and for patience. I didn't get any email notifications about that pull request until yours... though I did see it yesterday and thought it was closed.

I guess if the service launches sunshine as administrator, it'll have access to the configuration files specified and the patch will work good. I've merged that change. Thanks for the patch @psyke83 !

No problem, but it seems that my commit was lost after you rebased your ucrt64 branch against nightly?

[cgutman]

Is this still needed with the change to config.cpp?

[psyke83]

I think so?

Test 1:

• Delete existing config directory
• Freshly install Sunshine built from current PR branch
• Set port = 46989
• Stop sunshine service and relaunch from shortcut as non-admin

Result: Sunshine launches with the correct custom web UI port.

Test 2:

• Repeat test 1 with build that omits migrate-config.bat OR keep PR build+config from test 1 and remove Users read folder privilege manually.

Result: several second delay on "Waiting for Web UI to be ready....", and UI opens to the (incorrect) default port.

[tez011]

Now that C++20 is in, this can be considered.

What's the appetite for merging this environment change? If it's slim-to-none, I can try porting the WGC change to mingw64 now that msys2/MINGW-packages#20477 is in. I have a suspicion that MSVCRT and the UWP runtime won't play together well, so hoping to get some confirmation here before I spend time in that rabbit hole.

[ReenigneArcher]

What's the appetite for merging this environment change?

We'd like to move to UCRT64 since mingw seems to be considered legacy by the packaging team.