Valheim Dedicated Server Crashes Post Unity 2022 Update

[ejtejada]

Hello there,

I had been running Valheim dedicated server on Ampere arm based server, using the below install script:
https://gist.github.com/husjon/c5225997eb9798d38db9f2fca98891ef
This was working for quite some time.
However, as of a November 7th update of the server and clients to Unity 2022, the server will disconnect clients and then crash about 20 seconds after a client connects. I can confirm I am pulling and compiling a fairly recent version of box64:

box64 --version
Dynarec for ARM64, with extension: ASIMD AES CRC32 PMULL ATOMICS PageSize:4096 Running on Neoverse-N1 with 4 Cores
Params database has 30 entries
Box64 with Dynarec v0.2.5 2a4fe803 built on Dec 24 2023 07:41:33

Finally, I would like to submit logs, but the logfiles generated are over 350 MB in size. How would I submit them here?

I am generating these logs via
BOX64_LOG=2 BOX64_TRACE_FILE=valheim_arm_crash.txt /home/ubuntu/valheim_server/valheim_server.x86_64 -nographics - batchmode -port 2456 -public 1 -name serverName -password sometemporarydummypassword -savedir /home/ubuntu/valheim_data

Thank you for your time and for this awesome project.

[sea212]

Valheim server still works using Box64 v0.2.4 on my end (RPI4 4GB, Ubuntu 64bit), but fails to start using v0.2.6.

However, starting with v0.2.4 leads to a server crash after an arbitrary amount of time (leaving behind mono_crash memory dumps)

[ptitSeb]

@sea212 did you try with current version also? Can you bisect the issue if it's still broken?

[sea212]

@ptitSeb I'll provide more information shortly.

[sea212]

When tried to bisect I realized that I can't reliably reproduce the error that the server hangs on launch. Sometimes it does, most of the times it seems to work.

[ptitSeb]

If it's a random issue, you can try to start the server with BOX64_DYNAREC_STRONGMEM=1 (or play with ~/.box64rc if you know it), that might be a needed params.

[ejtejada]

If it's a random issue, you can try to start the server with BOX64_DYNAREC_STRONGMEM=1 (or play with ~/.box64rc if you know it), that might be a needed params.

To clarify, should I stay on a more recent built of box64, or revert to 0.2.4 before trying this flag?
Also, I would like to provide logs, but they are still huge with
BOX64_LOG=2 BOX64_TRACE_FILE=valheim_arm_crash.txt

Is there any way to make them smaller? Thanks again for the help

[ptitSeb]

If it's a random issue, you can try to start the server with BOX64_DYNAREC_STRONGMEM=1 (or play with ~/.box64rc if you know it), that might be a needed params.

To clarify, should I stay on a more recent built of box64, or revert to 0.2.4 before trying this flag?

Stay on current, it would be easier for me. Als, if it just work with BOX64_DYNARC_STRONGMEM=1 that means that would be the solution! The parameter could go in a config file so it's automaticaly applied.

Also, I would like to provide logs, but they are still huge with BOX64_LOG=2 BOX64_TRACE_FILE=valheim_arm_crash.txt

Is there any way to make them smaller? Thanks again for the help

You can use BOX64_ROLLING_LOG=1 instead, that would just show the last 10 function call before the crash. But if the issue is a multi-thread isseu, those logs will be hardly usefull anyway, as each crash would show something different...

[ejtejada]

@ptitSeb Ok,
I recompiled to make sure I am on the latest version

box64 --version
Dynarec for ARM64, with extension: ASIMD AES CRC32 PMULL SHA1 SHA2 PageSize:4096 Running on Neoverse-N1 with 4 Cores
Params database has 48 entries
Params database has 48 entries
Box64 with Dynarec v0.2.7 41bfd757 built on Jan  7 2024 20:31:39

I ran this below and was able to generate a more interesting log

BOX64_DYNARC_STRONGMEM=1 BOX64_ROLLING_LOG=1 BOX64_TRACE_FILE=valheim_arm_crash.txt /home/ubuntu/valheim_server/valheim_server.x86_64
\ -nographics -batchmode -port 2456 -public 1 -name serverName -password sometemporarydummypassword 
\ -savedir /home/ubuntu/valheim_data

valheim_arm_crash.txt

In particular

Warning: Global Symbol _ZTH15gDeferredAction not found, cannot apply R_X86_64_GLOB_DAT @0x7fff01d3d660 ((nil)) in /home/ubuntu/valheim_server/UnityPlayer.so
Error loading needed lib libpulse-mainloop-glib.so.0
Error loading one of needed lib
Error initializing needed lib /home/ubuntu/valheim_server/valheim_server_Data/Plugins/libparty.so

Which is odd, previously libpulse-mainloop was only needed for crossplay, so maybe the default behavior is now crossplay on?

[o-brkyr]

I'm on an RPI 5 running Ubuntu Server 23.10, getting the same issue as those above - silent crashing after an arbitrary period of time.

I'm running

Dynarec will try to emulate a strong memory model with limited performance loss
Dynarec for ARM64, with extension: ASIMD AES CRC32 PMULL SHA1 SHA2 PageSize:4096 Running on Cortex-A76 with 4 Cores
Params database has 48 entries
Box64 with Dynarec v0.2.7 328d5c62 built on Jan 11 2024 19:17:20

To note, I'm seeing occasional Error loading one of needed lib errors, such as:

Error loading needed lib libparty.so
Warning: Cannot dlopen("libparty.so"/0x71ab21f0, 101)

Using emulated ./linux64/steamclient.so
Redirecting overridden malloc from symtab function for ./linux64/steamclient.so
Warning: Weak Symbol _ITM_RU1 not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0c233fb8 (0x32a296)
Warning: Weak Symbol _ZGTtnam not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0c233fc0 (0x32a2a6)
Warning: Weak Symbol _ITM_memcpyRtWn not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0c233fc8 (0x32a2b6)
Warning: Weak Symbol _ITM_RU8 not found, cannot apply R_X86_64_JUMP_SLOT @0x7fff0c233fd0 (0x32a2c6)
[S_API] SteamAPI_Init(): Loaded local 'steamclient.so' OK.
Using native(wrapped) crashhandler.so
CAppInfoCacheReadFromDiskThread took 1 milliseconds to initialize
Error loading needed lib steamservice.so
Warning: Cannot dlopen("steamservice.so"/0xffff786ba8b0, 2)
dlmopen steamservice.so failed: Cannot dlopen("steamservice.so"/0xffff786ba8b0, 2)

Setting breakpad minidump AppID = 892970
SteamInternal_SetMinidumpSteamID:  Caching Steam ID:  76561197960265728 [API loaded no]
Error loading needed lib libsteam.so
Warning: Cannot dlopen("libsteam.so"/0x7fff07038ad9, 2)
[S_API FAIL] Tried to access Steam interface SteamNetworkingUtils004 before SteamAPI_Init succeeded.

Let me know if there's anything else I can provide.

[o-brkyr]

In addition to the above, I took a log of my last run of the game using the following shell script:

#!/bin/bash
export templdpath=$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=./linux64:$LD_LIBRARY_PATH
export SteamAppId=892970


export BOX64_SHOWBT=1
export BOX64_LOG=1
export BOX64_TRACE_FILE=log.txt
export BOX64_DYNAREC_STRONGMEM=1

echo "Starting server PRESS CTRL-C to exit"

# Tip: Make a local copy of this script to avoid it being overwritten by steam.
# NOTE: Minimum password length is 5 characters & Password cant be in the server name.
# NOTE: You need to make sure the ports 2456-2458 is being forwarded to your server through your local router & firewall.
./valheim_server.x86_64 \
-name "something" \
-port 2456 \
-world "something" \
-password "secret" \
-nographics \
-batchmode \
-public 0 \

export LD_LIBRARY_PATH=$templdpath

log.txt

Pardon my ignorance, but it's curious to see so many errors about libpulse-mainloop-glib.so.0 - If I run sudo apt install libpulse-mainloop-glib0 I'm apparently on the newest version. party.so straight up doesn't exist, but libparty.so does in /valheim_server_Data/Plugins. steamservice.so doesn't appear to exist on my system.

[nitroinferno]

I am having a similar issue. I do not have BOX64_DYNARC_STRONGMEM set to 1.
Details from box64 --version:
Dynarec for ARM64, with extension: ASIMD CRC32 PageSize:4096 Running on Cortex-A72 with 4 Cores
Params database has 49 entries
Box64 with Dynarec v0.2.7 57ca9df built on Jan 18 2024 22:03:56

Whats interesting is, sometimes the Valheim service will run fine for a bit (up to 8 hrs it ran once!), sometimes it will only live for roughly 5 minutes before silent failure.

Recently though, the Valheim service has been silently failing typically after an hour, usually sooner in recent days.. can't get it to run up to anywhere near 8 hours again...

If it would help for me to run and trace logs I can do so if it helps resolve this sooner. I am a novice at linux and bash however but I think I could handle running a similar script to that mentioned above.

[ptitSeb]

@nitroinferno what if you run with BOX64_DYNAREC_STRONGMEM=1? Does it make the server more stable?

[nitroinferno]

@ptitSeb How do I run the valheim service with BOX64_DYNAREC_STRONGMEM=1 from terminal? For example I run the server exec from valheim.service, and typically do 'systemctl start valheim' right in the terminal.

[ptitSeb]

@ptitSeb How do I run the valheim service with BOX64_DYNAREC_STRONGMEM=1 from terminal? For example I run the server exec from valheim.service, and typically do 'systemctl start valheim' right in the terminal.

The simpler would be to use either /etc/box64.box64rc or if the account used for the service has a home, create a .box64rc file in that home and put inside:

[valheim_server.x86_64]
BOX64_DYNAREC_STRONGMEM=1

And the parameter will automatically picked up.

[sea212]

@ptitSeb I already ran it using BOX64_DYNAREX_STRONGMEM=1. Inspecting the logs with and without STRONGMEM, I figured out box64 loads libmono which automatically toggles BIGBLOCK=0 and STRONGMEM=1. Inspecting the folder, it also becomes apparent that mono is involved in the sporadic crash, as there are mono memory dumps created when the server silently crashes (the process still runs, but it doesn't provide any of the expected functions).

[nitroinferno]

@ptitSeb How do I run the valheim service with BOX64_DYNAREC_STRONGMEM=1 from terminal? For example I run the server exec from valheim.service, and typically do 'systemctl start valheim' right in the terminal.

The simpler would be to use either /etc/box64.box64rc or if the account used for the service has a home, create a .box64rc file in that home and put inside:

[valheim_server.x86_64]
BOX64_DYNAREC_STRONGMEM=1

And the parameter will automatically picked up.

Thank you, I set it under the /etc/box64.box64rc, since the service runs from /etc/ directory. Seems the program is staying alive stable now (could also be luck of the draw). However, its been running for just a bit over 2 hours, which it hasn't done for awhile. I will continue to monitor it. Its been quite some time that it ran for this long without the silent failure.

To sea212's point, I do have mono_crash.mem files within ~/home//valheim_server/ directory. There are 7 files to be specific. Believe these are from the silent failures earlier in the day, I did not reboot since modifying the box64.box64rc

Update: Seems it was luck of the draw it ran 2 hours. After restarting it didn't stay online for longer than 5minutes, then subsequent restarts wouldn't last beyond 15m.

[nitroinferno]

Did a bit more digging - after my server ran for 40hours straight, no issue hosting up to 3 players at different times. Until silently crashing again at 40 hours, when no one was logged in. Since then it typically fails after 15-45min.

I went into the directory of the non-responsive PID ( /proc/$[PID] ) to do some investigation. I found the following:
Using cat /proc/$[PID]/wchan it returns: futex_wait_queue

Using Sudo cat /proc/$[PID]/syscall it returns something along the lines of:
98 0x150034780 0x80 0x153 0x10080e018 0x0 0x0 0x7fdcdc9a80 0x7facba3c28

This does not change, while the program is stuck.
it's always 98, and always 0x80 for third variable and also always has '0x10080e018 0x0 0x0'
Found within /usr/arm-linux-gnueabihf/include/asm-generic that 98 corresponds to:

#define __NR_futex 98
__SC_3264(__NR_futex, sys_futex_time32, sys_futex)
#endif

Using Sudo cat /proc/$[PID]/stack it always returns the following once the program is stuck:

[<0>] futex_wait_queue+0x78/0xac
[<0>] futex_wait+0x100/0x1c0
[<0>] do_futex+0xec/0x194
[<0>] __arm64_sys_futex+0x84/0x19c
[<0>] invoke_syscall+0x50/0x120
[<0>] el0_svc_common.constprop.0+0x68/0x124
[<0>] do_el0_svc+0x34/0xd0
[<0>] el0_svc+0x30/0x94
[<0>] el0t_64_sync_handler+0xf4/0x120
[<0>] el0t_64_sync+0x18c/0x190

This is all a bit over my head - but hoping this may help determine whether this is box64 related or not..

[nitroinferno]

I believe there may be 2 issues at hand. The crashes on initialization / startup sometimes generate the mono_crash.mem files (these crashes are RARE - at least in my case). I overclocked by rpi4 to 2GHz and it stopped any and all crashing on startup / initialization. I am suspecting they may be somehow tied to the lower end clock rate of the CPU? (I have no gauge whether this makes logical sense or not)

I created a service which monitors to see if the primary .x86_64 PID has become stuck in 'Sleeping' status as seen in htop. This is an easy work around to the sporadic failed starts.

My main issue is how to diagnose or provide useful info on when the main process gets stuck in 'sleeping' state. I tried running with BOX64_LOG=1 but I don't think it provided any useful info; it pretty much looked similar to #1182 (comment). I Tried running BOX64_LOG=2 and 3, and it both generated a log file that was enormous and it couldn't get through a startup / initialization routine. I believe the RPI4 isn't powerful enough to handle all this logging and initialization simultaneously.

From strace -c I get a lot of errors reported for futex, restart_syscall, and read.

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 52.30    0.062819         112       557        59 futex
 23.66    0.028420        1291        22           clock_nanosleep
 14.65    0.017595        2199         8         4 restart_syscall
  8.88    0.010666        2133         5           epoll_pwait
  0.26    0.000316          26        12           clock_gettime
  0.16    0.000193         193         1           sendto
  0.04    0.000053          13         4         4 read
  0.03    0.000037          37         1           recvfrom
  0.00    0.000003           1         2           pselect6
  0.00    0.000001           0         4           ppoll
------ ----------- ----------- --------- --------- ----------------
100.00    0.120103         194       616        67 total

I suspect the issue is a deadlock or resource contention issue causing the sporadic silent failures, where the main process cannot be returned to 'running' status after going to 'sleep'. Is there any BOX64 settings I can tweak to avoid this, or methods I can use to generate more useful log data?

note: I tried using gbd to get useful trace info logs both when running, and stuck in sleeping state. Not sure how useful these are I will attach one of each to this post.
stack_traceRun.txt
stack_tracescrash.txt

[ptitSeb]

Thanks for those analysis, that some interesting stats.

It seems this is some multitasking issues. My feeling is that BOX4_DYNAREC_STRONGMEM is the main parameter to tweek (values from 0 to 3, 0 is default). But BOX64_DYNAREC_BIGBLOCK might have some effect too (values from 0 to 3, 1 is default).
If MonoBleedingEdge is loaded, it will override those settings, so you need to use BOX64_DYNAREC_BLEEDING_EDGE =0 to disable this lib detection have have full control on the settings.
Use the ~/.box64rc or /etc/box64.box64rc file with the [valheim_server.x86_64] section has mentionned earlier for easy tweaking.

[husjon]

Hi, author of the guide in OP here.
Thank you all looking into this. :)

I've not had much time to look into this myself and I also brought down my original Ampere instance in OCI.
A month ago I decided to spin up an instance for Terraria and this week I repurposed it for Valheim again to try to help out figuring out this issue.

The instance was running the latest version of Valheim Valheim l-0.217.38 (network version 20) using Box64 v0.2.6.
No changes were done to environment variables used by Box64 or box64rc.
There were a couple of packages that were needed for Terraria specifically but unfortunately I did not manage to note down the exact ones prior to tearing it down to redo the install (one I do remember on top of my head were libcurl3-gnutls).

I'm in the process of spinning up a new OCI instance using the same specs and will do a clean install of Valheim to narrow down the exact packages.

Edit: I forgot to add that the Valheim server was running over night with no signs of issues (total runtime was about 20 hours prior to teardown).

Edit2: Instance is up and has been running for about 3 hours, no additional steps have been done other than running the installation script from my guide.
I will be letting the server run for a while, connect and play a bit over the weekend then monitor the logs over time to look for abnormalities.
Gist containing details about the server packages etc can be found here and will be updated over time https://gist.github.com/husjon/a94b6760e036e83d0b67a04e3916033d

Update 1: The server usually logs the amount of connections every 10 minutes.
ala: 02/09/2024 22:15:15: Connections 0 ZDOS:30349 sent:0 recv:0
After about 4 hours (18:41 - 22:40UTC) the server has deadlocked.
having to send SIGILL to get it to stop.
This does at the very least give me a baseline to work from.

Update 2: After restarting from the last run and letting it run over night the server ran for just over an hour and deadlocked with no players online.

Update 3: It seem like the packages I mentioned was a false report, I must have used them for something else. I installed the Terraria Server and it started without adding more packages than were already installed, in any case I will start looking at using box64rc to see how the Valheim server behaves.

[ebarrragn]

Hi All,
We have two servers running on a Raspberry Pi 5. They've been up for around 60 hours, and 3 players, with this config (and still going strong):

[valheim_server.x86_64]
BOX64_DYNAREC_STRONGMEM=3
BOX64_DYNAREC_BLEEDING_EDGE =0
BOX64_DYNAREC_BIGBLOCK=3

EDIT:
The server run for around 100 hours without any problem, and it has been downgraded to a Raspberry Pi 4, that has been running for 10 hours without an issue.

[nitroinferno]

Thanks for those analysis, that some interesting stats.

It seems this is some multitasking issues. My feeling is that BOX4_DYNAREC_STRONGMEM is the main parameter to tweek (values from 0 to 3, 0 is default). But BOX64_DYNAREC_BIGBLOCK might have some effect too (values from 0 to 3, 1 is default). If MonoBleedingEdge is loaded, it will override those settings, so you need to use BOX64_DYNAREC_BLEEDING_EDGE =0 to disable this lib detection have have full control on the settings. Use the ~/.box64rc or /etc/box64.box64rc file with the [valheim_server.x86_64] section has mentionned earlier for easy tweaking.

Similar to @ebarrragn I set the following:
Environment=BOX64_DYNAREC_STRONGMEM=3
Environment=BOX64_DYNAREC_BIGBLOCK=2
Environment=BOX64_DYNAREC_BLEEDING_EDGE=0

Servers been MUCH more stable and running for long hours on a pi4 4gb. ran for 43 hours the other day which it has never done before for me. I believe modifying the settings may be the solution. I will have to try with bigblock = 3.

[husjon]

I've been trying to run the server in OCI on Ampere trying out a variety of environment variables but not getting any positive results, using the following environment variables:

BOX64_LOG=1
BOX64_DYNAREC_BIGBLOCK=3
BOX64_DYNAREC_STRONGMEM=3
BOX64_DYNAREC_BLEEDING_EDGE=0
BOX64_TRACE_FILE=/home/ubuntu/valheim.box64.log

The tracefile is spammed of the following segfaults triggered by FillBlock:

FillBlock at 0x2fdd70 triggered a segfault, canceling
FillBlock triggered a segfault at 0x300000 from 0x35012ef4
FillBlock at 0x2fddbc triggered a segfault, canceling
FillBlock triggered a segfault at 0x300000 from 0x35012ef4
FillBlock at 0x2fddd4 triggered a segfault, canceling

https://gist.github.com/husjon/a94b6760e036e83d0b67a04e3916033d#file-valheim-box64-deadlock-01-log
It then refuses to continue or stop and I have to send SIGILL.
I'll continue to play around with the variables to see if I can get some more info.

Update:
Leaving only the following variables the server starts up normally

BOX64_LOG=1
BOX64_TRACE_FILE=/home/ubuntu/valheim.box64.log

normal startup log: https://gist.github.com/husjon/a94b6760e036e83d0b67a04e3916033d#file-valheim-box64-normal-log

[ptitSeb]

Make sure you are using latest version box64 also. Or at least the same version across those various tests.

[husjon]

I've been on v0.2.6 for all tests so far, but I'll take a look at building and testing towards latest on main a bit later.

[husjon]

After settings BLEEDING_EDGE=0 and STRONGMEM=3, the server have now been running for just over 20 hours (using Box64 v0.2.6).
I will let the server run for a while longer to see how it behaves.

Update 1: The server has now been running for close to 30 hours and still chugging along, still I feel it might be too early to say for sure so I'll continue to monitor the server.
Currently the server is using Steams multiplayer backend, but I'm curious to see how it would behave if Crossplay was enabled as that would use PlayFabs backend instead (this was a massive PITA when we tried getting it going before), I might spin up a separate instance for testing that at some point.

[Pohtaytoh]

My server is encountering a crash after 5-15mins which subsequently throws a "failed to connect" when trying to reconnect. We end up needing to restart the server to connect successfully only for it to eventually crash again. This was with just 1-2 people to test the variables.

I'm running the suggested variables mentioned above:

BOX64_DYNAREC_BIGBLOCK=3
BOX64_DYNAREC_STRONGMEM=3
BOX64_DYNAREC_BLEEDING_EDGE=0

[nitroinferno]

My server is encountering a crash after 5-15mins which subsequently throws a "failed to connect" when trying to reconnect. We end up needing to restart the server to connect successfully only for it to eventually crash again. This was with just 1-2 people to test the variables.

I'm running the suggested variables mentioned above:

BOX64_DYNAREC_BIGBLOCK=3
BOX64_DYNAREC_STRONGMEM=3
BOX64_DYNAREC_BLEEDING_EDGE=0

@Pohtaytoh Try setting the bigblock above to =2. Just a shot in the dark but mines been consistently working 40+ hours with it set to that with other settings above, though I'm hosting on a rpi4 without OCI ampere.

[nitroinferno]

@husjon hoping you can shed some light on the use of v0.2.7 eda857c or v0.2.7 eda857c. When you stated its still chugging along with respect to v0.2.7 eda857c, did it have faults as I do below? Most of these faults are due to mono lib it seems since these generate the mono-blob files.

I am still using v0.2.6 stable branch with the following params.

BOX64_LOG=1
BOX64_DYNAREC_BIGBLOCK=1
BOX64_DYNAREC_STRONGMEM=2
BOX64_DYNAREC_BLEEDING_EDGE=0

For whatever reason if I set BIGBLOCK to 0, it causes a lot of issues on a pi, I flagged multiple short duration faults on Apr 28th when I tried to use Bigblock=0. Either way for the most part it runs smooth anywhere from 12-24hrs. I am thinking about rolling it forward to v0.2.7 eda857c, however wasn't sure if that was less faulty with respect to mono related crashes, or whether v0.2.7 eda857c was either.

nitro@raspberrypi:~ $ journalctl -u valheim | grep -i exited
Mar 04 13:02:06 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
Mar 04 13:13:17 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
Mar 04 21:03:57 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
Mar 05 23:03:18 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 08 06:03:06 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 10 09:36:38 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 11 04:08:51 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 11 22:21:56 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 14 02:02:16 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 18 04:39:19 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 19 00:13:36 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 19 14:46:33 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 21 06:03:23 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 21 22:08:01 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 27 13:42:31 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Mar 28 08:59:51 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 29 16:48:01 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Mar 30 00:19:53 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 01 07:07:59 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 02 08:55:50 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 02 16:28:19 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 04 10:43:54 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 08 16:03:09 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 08 23:50:41 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 09 23:22:53 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 11 10:55:55 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 12 15:43:33 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 15 15:40:13 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 18 12:22:02 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 19 11:27:54 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
Apr 19 20:43:00 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 20 22:19:27 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 21 12:08:44 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 21 12:12:38 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 22 04:03:24 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 23 14:15:34 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 23 19:50:41 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 26 09:05:31 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 26 12:27:22 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 27 09:02:43 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 27 23:57:28 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
NOTE:~ $ Apr 28 20:32:54 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
NOTE:~ $ Apr 28 21:05:41 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
NOTE:~ $ Apr 28 21:09:31 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
NOTE:~ $ Apr 28 21:45:20 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=9/KILL
Apr 29 09:09:45 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
Apr 29 12:23:10 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
Apr 30 11:27:14 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=5/TRAP
May 01 19:19:13 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT
May 02 00:08:26 raspberrypi systemd[1]: valheim.service: Main process exited, code=killed, status=6/ABRT

[husjon]

@nitroinferno from what I've gathered from the previous comments, there is a quite a difference running it on OCI versus a Raspberry Pi.
BIGBLOCK seem to work fine on OCI Ampere instances, but not on a raspberry.

As for status on the server, it is still running.

ubuntu@game-server:~$ systemctl --user status valheim_server
● valheim_server.service - Valheim Dedicated Server
     Loaded: loaded (/home/ubuntu/.config/systemd/user/valheim_server.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-04-12 16:46:45 UTC; 2 weeks 6 days ago

I see you have a Raspberry Pi 4, so it will not be an exact match but I'll do some testing on my Raspberry Pi 5 (I unfortunately do not have a Pi 4 available) over the weekend.
(Side note: I am using the same build parameters as RPI4, -D RPI4ARM64=1 -D CMAKE_BUILD_TYPE=RelWithDebInfo since my RPI5 is not using 16K pages.)

[nitroinferno]

Oh wow 2 weeks uptime with no faults! With this I think ill roll my box64 forward to v0.2.7 eda857c. I'll Give a test run and report back.
Apparently I've read pi5 has its own share of issues as well outside pi4.

What's interesting is when I was using build: v0.2.7 57ca9df it ran best with BIGBLOCK=3, however v0.2.6 cannot use bigblock=3 at all. It will not even get through start sequence.

There were not a lot of crashes with v0.2.7 57ca9df, however it would instead silently lock up about the same amount: 12-24hr intervals.

[husjon]

@nitroinferno I've had the server running on my RPI 5 for 2 days now using eda857c
I haven't played on it, just let it run on its own.
I'm not sure if the issue you're having is an isolated case for RPI 4.
What are the specs of the RPI 4 you have?

My RPI 5 is the 4GB model.

pi@raspberrypi:~ $ box64 --version | grep built
Box64 with Dynarec v0.2.7 eda857cb built on May  2 2024 21:21:30

pi@raspberrypi:~ $ systemctl --user status valheim_server.service
● valheim_server.service - Valheim Dedicated Server
     Loaded: loaded (/home/pi/.config/systemd/user/valheim_server.service; enabled; preset: enabled)
     Active: active (running) since Thu 2024-05-02 22:54:20 CEST; 2 days ago

[nitroinferno]

Pi 4b rev1.2 4gb:
Mine has been up 3 days this current iteration but typically it faults near this or 2 day mark.

nitro@raspberrypi:~/box64 $ cat /proc/cpuinfo
processor       : 0
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
processor       : 1
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
processor       : 2
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
processor       : 3
BogoMIPS        : 108.00
Features        : fp asimd evtstrm crc32 cpuid
CPU implementer : 0x41
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0xd08
CPU revision    : 3
Revision        : c03112
Serial          : 100000008f6a4932
Model           : Raspberry Pi 4 Model B Rev 1.2

[husjon]

I was partially wrong in my previous comment.
It seem like the server have deadlocked since nothing has been logged to journalctl since May 3rd.

Actually after trying to connect to it, it seem like it is still running (to some degree), but is not able to handle connections.
I get the following entries in journalctl when trying to connect

May 03 12:25:06 raspberrypi valheim_server.x86_64[10338]: 05/03/2024 12:25:06: No autobackup needed yet...
May 03 12:26:09 raspberrypi valheim_server.x86_64[10338]: 05/03/2024 12:26:09:  Connections 0 ZDOS:81  sent:0 recv:0
May 03 12:36:09 raspberrypi valheim_server.x86_64[10338]: 05/03/2024 12:36:09:  Connections 0 ZDOS:81  sent:0 recv:0
May 03 12:46:09 raspberrypi valheim_server.x86_64[10338]: 05/03/2024 12:46:09:  Connections 0 ZDOS:81  sent:0 recv:0
May 05 15:22:44 raspberrypi valheim_server.x86_64[10338]: src/steamnetworkingsockets/clientlib/steamnetworkingsockets_connections.cpp (1115) : Assertion Failed: Unexpected auth avail -100 (Expired)
May 05 15:24:03 raspberrypi valheim_server.x86_64[10338]: src/steamnetworkingsockets/clientlib/steamnetworkingsockets_connections.cpp (1115) : Assertion Failed: Unexpected auth avail -100 (Expired)
May 05 15:24:03 raspberrypi valheim_server.x86_64[10338]: src/steamnetworkingsockets/clientlib/steamnetworkingsockets_connections.cpp (1115) : Assertion Failed: Unexpected auth avail -100 (Expired)

I'm gonna do an upgrade of the OS, reboot and restart the server, then let it run for a few more days.

[nitroinferno]

Ah yes this is a common occurrence I've come across with the rpi. I've chalked it up to the limited resources it has available. I created a service to detect and force a restart. Uses: [ps -o state -p $PID --no-headers] to get state, if its been S constantly for 60s its deadlocked. It only happens occasionally on v0.2.6 - its usually ABRT or TRAP signal kills (which i believe are the mono crashes). What's interesting is I noticed the deadlock was more prevelant on later builds - whereas the mono crashes were a lot less.

Somehow setting bigblock=3 significantly reduced the number of deadlocks in later builds - I have no idea why. I'll spin up a da857c later once I get a chance.

[husjon]

Seem like it deadlocked once again after about 30 hours on my RPI5.
I'll try to adjust some of the box64rc parameters to see if I can find something that works a bit better.

[spwly]

I am also trying to get a server working on a RPI5 (8GB) with box64. I tried a few of the settings for box64rc mentioned here but haven't been successful. Longest I had the server running before the silent crash was only an hour. Admittedly I can't figure out how to roll my box64 back to a different version so I have only been using the somewhat latest box64 v0.2.7 ecf1902 .

Appreciate all the work everyone has done so far - I will be sure to update with my settings if I get it stable on my system.

Update: Managed to install v0.2.6 e42001b which didn't seem to make a difference. Crashes pretty consistently after 5 minutes.

[husjon]

So far I've had the server running for about 36 hours with the problematic version mentioned above. (eda857c)
Box64 with Dynarec v0.2.7 eda857cb built on May 2 2024 21:21:30

I've been using the following settings:

[valheim_server.x86_64]
BOX64_DYNAREC_BLEEDING_EDGE=0
BOX64_DYNAREC_BIGBLOCK=0
BOX64_DYNAREC_STRONGMEM=2

BOX64_LOG=1
BOX64_TRACE_FILE=/home/pi/valheim.box64.log

@spwly it depends on how you installed box64, but assuming you used git, you'd need to check out the version / commit you're interested in, then do the cmake / make / make install procedure.
For example:

cd ~/box64/build
git fetch

git checkout <TAG / VERSION HERE> # (specific version: "v0.2.6" or commit "eda857cb")
cmake .. -DRPI4ARM64=1 -DCMAKE_BUILD_TYPE=RelWithDebInfo
make -j"2"

sudo make install

[spwly]

Many thanks @husjon - I got my version to eda857c . It didn't seem to affect my performance either, so I determined my issue must be elsewhere. Turns out, I hadn't been installing the box64rc file, and was only making changes to the file within /system. Properly installed with your settings, it has been up and running for over two hours now. I will let it run until I get a crash and report back with my total uptime.

Update: Must have jinxed it, crapped out right at 3 hours. Progress is progress though

[husjon]

Glad you got it running, but of course annoying that it didn't work after all :/
Just to make sure that it is picking up the environment variables (if you used the exact ones I mentioned), you should see the following lines shortly after starting it (only if LOG is set to 1 or higher).

May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_LOG=1
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_DYNAREC_BIGBLOCK=0
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_DYNAREC_STRONGMEM=2
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_DYNAREC_BLEEDING_EDGE=0
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_TRACE_FILE=/home/pi/valheim.box64.log

I have had some success as my server have been running for close to 3 days now using the settings mentioned in my previous comment (#1182 (comment)).
I will however do a restart of the valheim server and do another run to see if it was just a lucky run. :)

[husjon]

Almost 48 hours later and the server is still running normally on my RPI 5 using the settings mentioned above.

pi@raspberrypi:~ $ systemctl --user status valheim_server.service
● valheim_server.service - Valheim Dedicated Server
     Loaded: loaded (/home/pi/.config/systemd/user/valheim_server.service; enabled; preset: enabled)
     Active: active (running) since Sun 2024-05-12 22:56:45 CEST; 1 day 18h ago
   Main PID: 883789 (valheim_server.)
      Tasks: 41 (limit: 4387)
        CPU: 22h 54min 16.810s
     CGroup: /user.slice/user-1000.slice/user@1000.service/app.slice/valheim_server.service

I set up a similar monitoring solution as you @spwly but instead of restarting I just get a notification so that I can investigate it further. :/

One thing that I just started thinking about is, are you using an SD card as storage medium?
Mine is set up to use a USB stick, I'm using the Samsung BAR Plus USB 3.1 128GB (https://www.samsung.com/au/memory-storage/usb-flash-drive/usb-3-1-flash-drive-bar-plus-128gb-titanium-gray-muf-128be4-apc/)

[spwly]

Just to make sure that it is picking up the environment variables (if you used the exact ones I mentioned), you should see the following lines shortly after starting it (only if LOG is set to 1 or higher).

May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_LOG=1
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_DYNAREC_BIGBLOCK=0
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_DYNAREC_STRONGMEM=2
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_DYNAREC_BLEEDING_EDGE=0
May 09 23:28:28 raspberrypi valheim_server.x86_64[18757]: Applying BOX64_TRACE_FILE=/home/pi/valheim.box64.log

Thanks for mentioning this, because this was definitely not showing up for me. Apparently my box64rc settings were never being read. My issue was the file in the home directory, box64.box64rc needed to be renamed to just ".box64rc"!

With those parameters finally successfully set, I booted up the server last night after updating (new biome is out!) and played for a few hours with two other players. Everything ran smoothly and the server is still running 18 hours later.

I am running everything off the internal SD card, so if I get any more performance issues, I will try running off an SSD and see if it has any effect. I'm also curious if running the pi headless would speed things up.

[arthurgbranco]

Anyone not even able to connect to their server? My server is listed publicly and I've even used this node gamedig package to verify it's up but when I try to connect I get connection failed every time.

Server and box64 logs:
server_logs.txt
valheim.box64.log

[spwly]

@arthurgbranco I was having this issue for a while, found out my system had a firewall preinstalled on the OS that just needed to be disabled.

[arthurgbranco]

@spwly no firewall-cmd installed and I've added accepted ports to iptables, no idea what else I can do (it's debian).

[Cybercat87]

I have now also registered here because I am facing exactly the same problem and cannot get my Valheim server to run.

My setup: Raspberry Pi 5 8GB. I had managed to get the server running once with the instructions. This was done with the Raspberry Pi OS 64-bit.

However, I had the problem that the server did not run stably and shut down after a few minutes. Also, it never started automatically after a restart. Except for one time when the server couldn't be started manually anymore. Unfortunately, I don't have any log files here anymore.

Since then, I've been trying to restart the server somehow. I have now tried several operating systems through headless or whatever, but nothing has worked. It usually only loads as far as described by arthurgbranco in his log file, or ends with "Error loading needed lib libSDL3.so.0 Warning: Cannot dlopen("libSDL3.so.0"/0x7fff0a96150c, 2)".

I managed to start the server briefly via Ubuntu 23.10 according to the terminal (it showed how many players are online and what the IP address is), but unfortunately, nobody could log in here. After a restart of the Raspberry, however, this is no longer possible and the creation of the server ends again with the above-mentioned error.

[husjon]

After 10 days my server crashed on my RPI5 because of Mono but shortly after was restored by systemd.

This is using the configuration mentioned in my earlier comment #issuecomment-2105673700 with Box64 with Dynarec v0.2.7 eda857cb built on May 2 2024 21:21:30.

I will now be updating to the latest stable v0.2.8.

May 25 07:40:54 raspberrypi valheim_server.x86_64[1745875]: 05/25/2024 07:40:54:  Connections 0 ZDOS:16879  sent:0 recv:0
May 25 07:50:54 raspberrypi valheim_server.x86_64[1745875]: 05/25/2024 07:50:54:  Connections 0 ZDOS:16879  sent:0 recv:0
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         Native Crash Reporting
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: Got a SIGSEGV while executing native code. This usually indicates
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: a fatal error in the mono runtime or one of the native libraries
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: used by your application.
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         Native stacktrace:
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         0x3f031151fe - /home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so : mono_breakpoint_clean_code
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         0x3f030bded9 - /home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so : mono_unity_backtrace_from_context
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         0x3f03110bdd - /home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so : mono_breakpoint_clean_code
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         0x30333 - Unknown
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]:         Telemetry Dumper:
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: =================================================================
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: Pkilling 0x547717140224x from 0x547078532864x
May 25 07:56:29 raspberrypi valheim_server.x86_64[293878]: Could not exec mono-hang-watchdog, expected on path '/home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/etc/../bin/mono-hang-watchdog' (errno 2)
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: Pkilling 0x548053118720x from 0x547078532864x
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: Pkilling 0x548155813632x from 0x547078532864x
May 25 07:56:29 raspberrypi valheim_server.x86_64[1745875]: Pkilling 0x548240994368x from 0x547078532864x
May 25 07:56:29 raspberrypi systemd[805]: valheim_server.service: Main process exited, code=killed, status=6/ABRT
May 25 07:56:34 raspberrypi systemd[805]: valheim_server.service: Failed with result 'signal'.
May 25 07:56:34 raspberrypi systemd[805]: valheim_server.service: Consumed 5d 10min 7.530s CPU time.
May 25 07:56:39 raspberrypi systemd[805]: valheim_server.service: Scheduled restart job, restart counter is at 2.
May 25 07:56:39 raspberrypi systemd[805]: Stopped valheim_server.service - Valheim Dedicated Server.
May 25 07:56:39 raspberrypi systemd[805]: valheim_server.service: Consumed 5d 10min 7.530s CPU time.
May 25 07:56:39 raspberrypi systemd[805]: Started valheim_server.service - Valheim Dedicated Server.

[nitroinferno]

Attempted to update box64 to v0.2.9 b5c0a85 and now it won't take any parameters set either within /etc/box64.box64rc or ~/.box64rc. At a complete loss why it can't detect either.

Each has:

[valheim_server.x86_64]
BOX64_DYNAREC_STRONGMEM=2
BOX64_DYNAREC_BIGBLOCK=1
BOX64_DYNAREC_BLEEDING_EDGE=0

Rolled it back to v0.2.6 - still won't recognize the box64rc setting files now for some reason.. It was working before on v0.2.6 prior to update to v0.2.9..

I have now completely removed box64 using make uninstall, and deleting the box64 directory including the .git hidden files. Performed a clone of v0.2.8 and built it - it still cannot pickup the /etc/box64.box64rc configuration file parameters above. I am just completely baffled - how can it be ignoring the configuration files?

update:
Rolled it to eda857c using steps in #1182 (comment). Still not be detecting the configuration file - shouldn't it have built based on the repo at this commit where this was working??

update 2:
So I went ahead an launched the server from start_server.sh and it showed the following code below.. All of the code prior to [UnitMemory] does not get written to journalctl logs - or anywhere for that matter when launching valheim from it's respective service file. I do have a command to direct the normal valheim logs out to a log file -logFile "/home/nitro/valheim_server/logs/servlog.log" however this part hasn't been modified at all since switching box64 versions. However, the info isn't in the log file either, or journalctl which is strange. Perhaps something broke / bugged with my pi's journalctl logging after changing box64 versions.

Dynarec for ARM64, with extension: ASIMD CRC32 PageSize:4096 Running on Cortex-A72 with 4 Cores
Will use Hardware counter measured at 54.0 MHz emulating 864 MHz
Params database has 64 entries
Warning, unsupported Environment=BOX64_DYNAREC_BLEEDING_EDGE=0 for [valheim.service] in /home/nitro/.box64rc
Params database has 65 entries
Box64 with Dynarec v0.2.7 eda857cb built on May 26 2024 13:25:29
BOX64: Didn't detect 48bits of address space, considering it's 39bits
Counted 28 Env var
BOX64 LIB PATH: ./linux64/:./:lib/:lib64/:x86_64/:bin64/:libs64/:/lib/x86_64-linux-gnu/:/usr/lib/x86_64-linux-gnu/
BOX64 BIN PATH: ./:bin/:/usr/local/sbin/:/usr/local/bin/:/usr/sbin/:/usr/bin/:/sbin/:/bin/:/usr/local/games/:/usr/games/
Looking for ./valheim_server.x86_64
Apply RC params for valheim_server.x86_64
Applying BOX64_LOG=1
Applying BOX64_DYNAREC_BIGBLOCK=1
Applying BOX64_DYNAREC_STRONGMEM=2
Applying BOX64_DYNAREC_BLEEDING_EDGE=0
Applying BOX64_TRACE_FILE=/home/nitro/valheim.box64.log
BOX64 Trace redirected to "/home/nitro/valheim.box64.log"
[UnityMemory] Configuration Parameters - Can be set up in boot.config

Update 3: SOLVED - journalctl had some corrupted log files. Seems the journalctl service was bugged. restarted the system journalctl service and removed the log file switch case in the execstart of valheim server service -logFile "/home/nitro/valheim_server/logs/servlog.log"

[nitroinferno]

Server runs fantastic on eda857c, however something very odd happened today.

May 29 10:01:27 raspberrypi valheim_server.x86_64[562131]: Am I Host? True
May 29 10:01:45 raspberrypi valheim_server.x86_64[562131]: 05/29/2024 10:01:45:  Connections 0 ZDOS:977551  sent:0 recv:0
May 29 10:08:20 raspberrypi systemd[1]: valheim.service: Deactivated successfully.
May 29 10:08:20 raspberrypi systemd[1]: valheim.service: Consumed 3d 6h 25min 34.169s CPU time.

It was 'de-activated' randomly. It did not crash, nor did it restart after this 'de-activation. I was not logged into the game, nor home when this occurred. There is nothing else at this time in journalctl logs.

[husjon]

After a week my RPI5 server finally deadlocked using v0.2.8.

.box64rc

BOX64_LOG=1
BOX64_DYNAREC_BIGBLOCK=0
BOX64_DYNAREC_STRONGMEM=2
BOX64_DYNAREC_BLEEDING_EDGE=0
BOX64_TRACE_FILE=/home/pi/valheim.box64.log

On startup it did complain about missing DLL, as we found during testing v0.2.8 with crossplay was that libpulse-mainloop-glib0 needed to be updated.
I'm gonna install it from the repo and give it another shot.
Server is currently and will for the next run also run with crossplay disabled.

Update 1:
After installing libpulse-mainloop-glib0 (sudo apt install libpulse-mainloop-glib0), the mentioned error is no longer happening.

Update 2 - 2024.06.18:
Server had restarted by itself after crashing after a week due to mono.
Better than a deadlock.
I've set up Telegraf to log details about cpu, memory and io.

Jun 18 11:23:55 raspberrypi valheim_server.x86_64[858]: 06/18/2024 11:23:55: No autobackup needed yet...
Jun 18 11:23:56 raspberrypi valheim_server.x86_64[858]: Am I Host? True
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         Native Crash Reporting
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: Got a SIGSEGV while executing native code. This usually indicates
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: a fatal error in the mono runtime or one of the native libraries
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: used by your application.
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         Native stacktrace:
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         0x3f031151fe - /home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so : mono_break>
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         0x3f030bded9 - /home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so : mono_unity>
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         0x3f03110bdd - /home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/x86_64/libmonobdwgc-2.0.so : mono_break>
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         0x30333 - Unknown
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]:         Telemetry Dumper:
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: =================================================================
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: Pkilling 0x546971475712x from 0x546404822784x
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: Pkilling 0x547787309120x from 0x546404822784x
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: Pkilling 0x547695058688x from 0x546404822784x
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: Pkilling 0x547654319872x from 0x546404822784x
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[2364334]: Could not exec mono-hang-watchdog, expected on path '/home/pi/valheim_server/valheim_server_Data/MonoBleedingEdge/etc/.>
Jun 18 11:24:59 raspberrypi valheim_server.x86_64[858]: Entering thread summarizer pause from 0x546404822784x
Jun 18 11:24:59 raspberrypi systemd[817]: valheim_server.service: Main process exited, code=killed, status=6/ABRT
Jun 18 11:24:59 raspberrypi systemd[817]: valheim_server.service: Killing process 2364335 (valheim_server.) with signal SIGKILL.
Jun 18 11:24:59 raspberrypi systemd[817]: valheim_server.service: Failed with result 'signal'.
Jun 18 11:24:59 raspberrypi systemd[817]: valheim_server.service: Consumed 4d 1h 4min 54.026s CPU time.

[ishaiaa]

I'm running a valheim server on my Raspberry Pi 5 (8gb model, booted from USB stick), Ubuntu 23.10, v0.2.9 of box64
The server usually crashes after about a day.
Installing libpulse-mainloop-glib0 changed the server behavior for me aswell, now server crashes completly, instead of silent-crashing.
The server has crashed a few times when i was online, always around the time that the world was saving.
Sometimes it happens just after the world save, sometimes right before it.
I'm not sure if it's just a coincidence or it has something to do with the crash

[husjon]

Followup on my previous comment (#issuecomment-2158845248), server had crashed and restarted over the weekend, unfortunately the information from Telegraf / Influx didn't yield any news, the only thing that oculd be seen was a slight spike in cpu usage shortly after the crash which is from when the server starts up again.
I'm going to look into adjusting the box64 environment variables.

[husjon]

@ishaiaa what box64rc settings are you using if any?
If none, could you try the settings I mention here #issuecomment-2158845248

[ishaiaa]

@husjon yes, I'm using those settings, I forgot to mention that

[ishaiaa]

Update
It looks like it really has something to do with autosaving.
I used -saveinterval parameter in the server startup script, to modify autosave interval.
If i set it to 30 seconds, the server usually crashes after 2-6 hours (it was ~24-30h with default 30 minutes saveinterval)
Now I have set it to 10 years (as there is no way of disabling it entirely afaik), and the server is still running after 5 days with no errors

[CleytonKe]

Any solution? I'm using an Orange Pi 3 and I noticed that the crash is related to the auto backup. If you disable it by passing 0, the crash that occurred in minutes goes to 3-5 hours.

I'm using the latest version box86 and box64.