Steam Client crashes processing downloaded pre-cached shaders from Steam Cloud when AMD GPU is used

[user111222333]

Hello developer team of the Steam Client.
I am a user of the Steam Client on a GNU/Linux based host.
I am experiencing an issue described below.

Please review the following information:

Host info:

CPU: Info: 6-Core model: Intel Core i5-9400F bits: 64 type: MCP L2 cache: 9 MiB

Graphics: Device-1: Advanced Micro Devices [AMD/ATI] Ellesmere [Radeon RX 580] driver: amdgpu v: kernel
Display: wayland server: X.Org 1.20.11
OpenGL: renderer: Radeon RX 580 Series (POLARIS10 DRM 3.40.0 5.10.0-11-amd64 LLVM 11.0.1) v: 4.6 Mesa 20.3.5
resolution: 3840x2160~60Hz *Note: when gaming, I set resolution to 1080p.
Memory: GPU-VRAM: 8GB
Memory: RAM: total: 15.56 GiB

System: Kernel: 5.10.0-11-amd64 x86_64 bits: 64 Desktop: GNOME 3.38.6 Distro: Debian GNU/Linux 11 (bullseye)
Gnome-UI: 3.38.6

Steam Client: built: Feb 15, 2022
Steam API: v020
Steam package version: 1644966673

The issue:
Steam client connects to the Valve's cloud back-end and downloads pre-cached shaders for a number of games.
Specifically 'Rise of the Tomb Raider' and 'Metro Exodus'.
Once download ~300MB, 'UPDATING' task completed, 'Installation' task of the shaders content fails.
This info reflected under: LIBRARY->DOWNLOADING status screen.
During the installation of the downloaded shader pre-cached content, Steam Client shows progress reaching 93% relatively quickly, than pausing at 73% to 93%.
Steam client appears to use one CPU core at 100% load, for ~ 8 minutes, flowed by abnormal exit (crash) with no messages shown in GNOME UI, and with the system log message:

Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] Startup - updater built Feb 15 2022 22:36:44
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] Opted in to client beta 'publicbeta' via beta file
Feb 16 18:18:04 steamPC steam.desktop[33613]: You are in the 'publicbeta' client beta.
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] Loading cached metrics from disk (/home/user/.steam/debian-installation/package/steam_client_metrics.bin)
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] Using the following download hosts for Public, Realm steamglobal
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] 1. https://cdn.akamai.steamstatic.com, /client/, Realm 'steamglobal', weight was 100, source = 'update_hosts_cached.vdf'
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] 2. https://cdn.cloudflare.steamstatic.com, /client/, Realm 'steamglobal', weight was 100, source = 'update_hosts_cached.vdf'
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] 3. http://media.steampowered.com, /client/, Realm 'steamglobal', weight was 1, source = 'baked in'
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] Verifying installation...
Feb 16 18:18:04 steamPC steam.desktop[33613]: [2022-02-16 17:53:36] Verification complete
Feb 16 18:18:04 steamPC steam.desktop[33613]: src/common/pipes.cpp (883) : fatal stalled cross-thread pipe.
Feb 16 18:18:04 steamPC steam.desktop[33613]: src/common/pipes.cpp (883) : fatal stalled cross-thread pipe.
Feb 16 18:18:04 steamPC steam.desktop[33613]: src/common/pipes.cpp (883) : Fatal assert; application exiting
Feb 16 18:18:04 steamPC steam.desktop[33613]: src/common/pipes.cpp (883) : Fatal assert; application exiting
Feb 16 18:18:04 steamPC steam.desktop[33613]: Installing breakpad exception handler for appid(steam)/version(1644966673)
Feb 16 18:18:05 steamPC steam.desktop[36143]: assert_20220216181804_69.dmp[36143]: Uploading dump (out-of-process)
Feb 16 18:18:05 steamPC steam.desktop[36143]: /tmp/dumps/assert_20220216181804_69.dmp
Feb 16 18:18:05 steamPC assert_20220216181804_69.dmp[36143]: Uploading dump (out-of-process)#12/tmp/dumps/assert_20220216181804_69.dmp
Feb 16 18:18:05 steamPC steam.desktop[33613]: _ExitOnFatalAssert
Feb 16 18:18:06 steamPC steam.desktop[36143]: assert_20220216181804_69.dmp[36143]: Finished uploading minidump (out-of-process): success = yes
Feb 16 18:18:06 steamPC steam.desktop[36143]: assert_20220216181804_69.dmp[36143]: response: CrashID=bp-8521d09c-fe1b-4aea-a961-e6fa92220216
Feb 16 18:18:06 steamPC steam.desktop[36143]: assert_20220216181804_69.dmp[36143]: file ''/tmp/dumps/assert_20220216181804_69.dmp'', upload yes: ''CrashID=bp-8521d09c-fe1b-4aea-a961-e6fa92220216''
Feb 16 18:18:06 steamPC assert_20220216181804_69.dmp[36143]: Finished uploading minidump (out-of-process): success = yes
Feb 16 18:18:06 steamPC assert_20220216181804_69.dmp[36143]: response: CrashID=bp-8521d09c-fe1b-4aea-a961-e6fa92220216
Feb 16 18:18:06 steamPC assert_20220216181804_69.dmp[36143]: file ''/tmp/dumps/assert_20220216181804_69.dmp'', upload yes: ''CrashID=bp-8521d09c-fe1b-4aea-a961-e6fa92220216''
Feb 16 18:18:06 steamPC systemd[1062]: app-gnome-steam-33455.scope: Succeeded.
Feb 16 18:18:06 steamPC systemd[1062]: app-gnome-steam-33455.scope: Consumed 13min 26.946s CPU time.

As you can see, system logs /var/log/syslog shows that Steam App crashed, de-bug dump was captured and uploaded to the Valves servers with a unique ID:
upload yes: ''CrashID=bp-8521d09c-fe1b-4aea-a961-e6fa92220216'

When Steam Client re-launched, the sequence of event's described above repeats itself in the exact same manner.
Steam Client begins downloading shaders all over again, fails to install them and crashes.

Work around-1:
In order to continue to use Steam Client, I had to manually disable 'Shader Pre-Caching' option in Steam->Settings->Shader Pre-Caching

Work around-2:
Take AMD GPU out of the system, and replace it with Nvidia GPU governed by the proprietary driver blob from Nvidia. With Nvidia GPU + driver, none of the above issues present and Steam Client operates as expected.

Findings:
Downloaded pre-cached shaders are not processed by the Steam Client, resulting in:
'steam.desktop[33613]: src/common/pipes.cpp (883) : fatal stalled cross-thread pipe.' process crashing, triggering a subsequent Steam Client execution to abort.

QUESTION TIME:
Is there an underlying issue with the Steam Client <-> Steam Cloud bug in delivering pre-cached shaders mechanism for users utilizing an AMD based GPU?
Is there an issue with the host configuration, missing components or proper configuration in order to complete processing of the downloaded pre-cached shaders data when AMD-GPU is used?
Could some one from the dev-team to advise what can be done to re-solve this issue, or what might be the cause of such failure.

If more details needed, please reach out to me, I will do my best to help to diagnose this issue.
Thank you dev-team for your work and effort to bring Steam Client gaming experience to user on a GNU/Linux platforms.

DM

[fkleon]

I'm experiencing the same issue on a similar system but with the stable steam client:

CPU
    CPU Brand:  Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz

OS
    Operating System Version: Debian GNU/Linux 11 (bullseye) (64 bit)
    Kernel Version:  5.10.0-12-amd64
    X Window Manager:  GNOME Shell
    Steam Runtime Version:  steam-runtime_0.20220119.0

Video
    Driver:  AMD Radeon RX 580 Series (POLARIS10, DRM 3.40.0, 5.10.0-12-amd64, LLVM 11.0.1)
    Driver Version:  4.6 (Compatibility Profile) Mesa 20.3.5

This happens pretty reliably when lots of shader pre-caching downloads are scheduled; and usually when it processes the bigger updates, towards the end (90%+), in my case often for Borderlands 3.

Symptoms are the Steam UI freezing. It sometimes recovers after a while, and sometimes crashes steam.

Logs:

Mar 26 15:12:54 steam.desktop[119724]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Mar 26 15:12:54 steam.desktop[119724]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Mar 26 15:12:54 steam.desktop[119724]: src/common/pipes.cpp (655) : Fatal assert; application exiting
Mar 26 15:12:54 steam.desktop[119724]: src/common/pipes.cpp (655) : Fatal assert; application exiting
Mar 26 15:12:54 steam.desktop[119724]: Installing breakpad exception handler for appid(steam)/version(1647446817)
Mar 26 15:12:54 steam.desktop[198289]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Mar 26 15:12:54 steam.desktop[198289]: src/common/pipes.cpp (655) : Fatal assert; application exiting
Mar 26 15:12:54 steam.desktop[198289]: Installing breakpad exception handler for appid(gameoverlayui)/version(1.0)
Mar 26 15:12:54 steam.desktop[222927]: assert_20220326151254_7.dmp[222927]: Uploading dump (out-of-process)
Mar 26 15:12:54 steam.desktop[222927]: /tmp/dumps/assert_20220326151254_7.dmp
                                                     /tmp/dumps/assert_20220326151254_7.dmp
Mar 26 15:12:54 assert_20220326151254_7.dmp[222927]: Uploading dump (out-of-process)
Mar 26 15:12:54 steam.desktop[119724]: _ExitOnFatalAssert
Mar 26 15:12:54 steam.desktop[222929]: assert_20220326151254_679.dmp[222929]: Uploading dump (out-of-process)
Mar 26 15:12:54 steam.desktop[222929]: /tmp/dumps/assert_20220326151254_679.dmp
                                                       /tmp/dumps/assert_20220326151254_679.dmp
Mar 26 15:12:54 assert_20220326151254_679.dmp[222929]: Uploading dump (out-of-process)
                                                     /tmp/dumps/assert_20220326151254_3.dmp
Mar 26 15:12:54 assert_20220326151254_3.dmp[222936]: Uploading dump (out-of-process)
Mar 26 15:12:55 steam.desktop[222927]: assert_20220326151254_7.dmp[222927]: Finished uploading minidump (out-of-process): success = yes
Mar 26 15:12:55 steam.desktop[222927]: assert_20220326151254_7.dmp[222927]: response: Discarded=1
Mar 26 15:12:55 steam.desktop[222927]: assert_20220326151254_7.dmp[222927]: file ''/tmp/dumps/assert_20220326151254_7.dmp'', upload yes: ''Discarded=1''
Mar 26 15:12:55 assert_20220326151254_7.dmp[222927]: Finished uploading minidump (out-of-process): success = yes
Mar 26 15:12:55 assert_20220326151254_7.dmp[222927]: response: Discarded=1
Mar 26 15:12:55 assert_20220326151254_7.dmp[222927]: file ''/tmp/dumps/assert_20220326151254_7.dmp'', upload yes: ''Discarded=1''
Mar 26 15:12:55 steam.desktop[222929]: assert_20220326151254_679.dmp[222929]: Finished uploading minidump (out-of-process): success = yes
Mar 26 15:12:55 steam.desktop[222929]: assert_20220326151254_679.dmp[222929]: response: CrashID=bp-cbda0afc-a63e-49d1-b4e6-5ab2d2220325
Mar 26 15:12:55 steam.desktop[222929]: assert_20220326151254_679.dmp[222929]: file ''/tmp/dumps/assert_20220326151254_679.dmp'', upload yes: ''CrashID=bp-cbda0afc-a63e-49d1-b4e6-5ab2d2220325''
Mar 26 15:12:55 assert_20220326151254_679.dmp[222929]: Finished uploading minidump (out-of-process): success = yes
Mar 26 15:12:55 assert_20220326151254_679.dmp[222929]: response: CrashID=bp-cbda0afc-a63e-49d1-b4e6-5ab2d2220325
Mar 26 15:12:55 assert_20220326151254_679.dmp[222929]: file ''/tmp/dumps/assert_20220326151254_679.dmp'', upload yes: ''CrashID=bp-cbda0afc-a63e-49d1-b4e6-5ab2d2220325''
Mar 26 15:12:55 assert_20220326151254_3.dmp[222936]: Finished uploading minidump (out-of-process): success = yes
Mar 26 15:12:55 assert_20220326151254_3.dmp[222936]: response: Discarded=1

[user111222333]

This issue persist on a current steam client for linux:
Steam client application:
Built: Mar 31 2022, at 23:11:43
Steam API: v020
Steam package version: 1648789923

When client downloads pre-cached shaders for Metro Exodus, during the 'installation' phase. steam client still crashing with the following messages in the system logs '/var/log/syslog' ->

Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Fatal assert; application exiting
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Fatal assert; application exiting
Apr 5 22:32:06 steam-pc steam.desktop[21102]: Installing breakpad exception handler for appid(steam)/version(1648789923)
Apr 5 22:32:06 steam-pc steam.desktop[22094]: assert_20220405223206_71.dmp[22094]: Uploading dump (out-of-process)
Apr 5 22:32:06 steam-pc steam.desktop[22094]: /tmp/dumps/assert_20220405223206_71.dmp
Apr 5 22:32:06 steam-pc assert_20220405223206_71.dmp[22094]: Uploading dump (out-of-process)#12/tmp/dumps/assert_20220405223206_71.dmp
Apr 5 22:32:06 steam-pc steam.desktop[21102]: _ExitOnFatalAssert
Apr 5 22:32:08 steam-pc steam.desktop[22094]: assert_20220405223206_71.dmp[22094]: Finished uploading minidump (out-of-process): success = yes
Apr 5 22:32:08 steam-pc steam.desktop[22094]: assert_20220405223206_71.dmp[22094]: response: CrashID=bp-ba6f5e2d-dd84-4b8d-b6d4-369062220405
Apr 5 22:32:08 steam-pc steam.desktop[22094]: assert_20220405223206_71.dmp[22094]: file ''/tmp/dumps/assert_20220405223206_71.dmp'', upload yes: ''CrashID=bp-ba6f5e2d-dd84-4b8d-b6d4-369062220405''
Apr 5 22:32:08 steam-pc assert_20220405223206_71.dmp[22094]: Finished uploading minidump (out-of-process): success = yes
Apr 5 22:32:08 steam-pc assert_20220405223206_71.dmp[22094]: response: CrashID=bp-ba6f5e2d-dd84-4b8d-b6d4-369062220405
Apr 5 22:32:08 steam-pc assert_20220405223206_71.dmp[22094]: file ''/tmp/dumps/assert_20220405223206_71.dmp'', upload yes: ''CrashID=bp-ba6f5e2d-dd84-4b8d-b6d4-369062220405''
Apr 5 22:32:08 steam-pc systemd[1079]: app-gnome-steam-20944.scope: Succeeded.
Apr 5 22:32:08 steam-pc systemd[1079]: app-gnome-steam-20944.scope: Consumed 10min 19.132s CPU time.

The key message here is:
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Stalled cross-thread pipe
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Fatal assert; application exiting
Apr 5 22:32:06 steam-pc steam.desktop[21102]: src/common/pipes.cpp (655) : Fatal assert; application exiting

Steam dev-team. could some one please review this issue and let us know what can be done to metigate it?

Thank you.