Raspberry pi Segmentation fault when compiled with LuaJIT

[gwech]

Working settings:

Minetest 5.6.1 (Linux)
Using Irrlicht 1.9.0mt8
Using Lua 5.1.5
BUILD_TYPE=Release
RUN_IN_PLACE=1
USE_CURL=1
USE_GETTEXT=0
USE_SOUND=1
STATIC_SHAREDIR="."

Crash constently :

Minetest 5.6.1 (Linux)
Using Irrlicht 1.9.0mt8
Using LuaJIT 2.1.0-beta3
BUILD_TYPE=Release
RUN_IN_PLACE=1
USE_CURL=1
USE_GETTEXT=0
USE_SOUND=1
STATIC_SHAREDIR="."

Recompiled for debug :

Minetest 5.6.1-debug (Linux)
Using Irrlicht 1.9.0mt8
Using LuaJIT 2.1.0-beta3
BUILD_TYPE=Debug
RUN_IN_PLACE=1
USE_CURL=1
USE_GETTEXT=0
USE_SOUND=1
STATIC_SHAREDIR="."

After running:

gdb -q -ex 'set confirm off' -ex 'r' -ex 'bt' -ex 'quit' --args ./minetest/bin/minetestserver --world world

I get:

Reading symbols from ./minetest/bin/minetestserver...
Starting program: /home/gwen/minetest/bin/minetestserver --world world
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ff68cd6c0 (LWP 8770)]
[New Thread 0x7ff60cc6c0 (LWP 8771)]

Thread 1 "minetestserver" received signal SIGSEGV, Segmentation fault.
0x0000007ff80a72a4 in ?? ()
#0 0x0000007ff80a72a4 in ?? ()
#1 0x000000000023ebd8 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

The config use is a minimal "Minetest Game" with no mod activated.
Once in a while it runs for few minutes.
When LuaJIT is not used, it is stable but a lot slower :)

I hope there is enough info here to get help.
Thanks

##### OS / Hardware
<!-- General information about your hardware and operating system -->
Operating system: 
Linux version 5.15.61-v8+ (dom@buildbot) (aarch64-linux-gnu-gcc-8 (Ubuntu/Linaro 8.4.0-3ubuntu1) 8.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #1579 SMP PREEMPT Fri Aug 26 11:16:44 BST 2022
CPU:
Hardware        : BCM2835
Revision        : c03111
Serial          : 100000001cd83ca1
Model           : Raspberry Pi 4 Model B Rev 1.1
MemTotal        : 3884404 kB

##### Summary
 Raspberry pi Segmentation fault when compiled with LuaJIT

##### Steps to reproduce
Compile minetestserver 5.6 Stable with  LuaJIT on Raspberry Pi 4

[gwech]

another detail that might be of insterest, I boot the raspberry off a USB HDD.
Not sure if this can have an impact.

[cyberarm]

I'm also getting a segfault trying to run a minetest server with LuaJIT enabled on a Pi 4 (from an SD card, for now)

Built 5.6.1 using:
cmake . -DRUN_IN_PLACE=0 -DBUILD_CLIENT=0 -DBUILD_SERVER=1 -DIRRLICHT_INCLUDE_DIR=../irrlicht/include

-- *** Will build version 5.6.1 ***
-- Using user-provided IrrlichtMt at subdirectory 'lib/irrlichtmt'
-- Using GMP provided by system.
-- Using JsonCpp provided by system.
-- Using LuaJIT provided by system.
-- cURL support enabled.
-- GetText disabled.
-- ncurses console enabled.
-- PostgreSQL backend enabled
-- PostgreSQL includes: /usr/include/postgresql
-- LevelDB not found!
-- Redis not found!
-- Prometheus client disabled.
-- SpatialIndex not found!
-- Looking for ZSTD_initCStream
-- Looking for ZSTD_initCStream - found
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE) 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/cyberarm/Code/minetest

Crash Dump:

Reading symbols from /home/cyberarm/Code/minetest/bin/minetestserver...
(No debugging symbols found in /home/cyberarm/Code/minetest/bin/minetestserver)
Starting program: /home/cyberarm/Code/minetest/bin/minetestserver --world /home/cyberarm/MineTest/ --config /home/cyberarm/MineTest/minetest.conf
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
[New Thread 0xfffff68106c0 (LWP 431131)]
[New Thread 0xfffff600f6c0 (LWP 431132)]
[MOD] Bees Loaded
[OK] Mesecons
[MOD] Mobs Redo loaded
[MOD] Mobs Redo Animals loaded
[MOD] Mob Horse loaded
[UnifiedDyes] Loaded!
[New Thread 0xffffefffc6c0 (LWP 431157)]
[New Thread 0xffffedbfa6c0 (LWP 431158)]
[New Thread 0xffffed3f96c0 (LWP 431159)]
[New Thread 0xffffecbf86c0 (LWP 431160)]
[New Thread 0xffffdbffc6c0 (LWP 431161)]
Unified Inventory. Inventory size: 950
[Thread 0xffffdbffc6c0 (LWP 431161) exited]
[New Thread 0xffffdbffc6c0 (LWP 431162)]
[Thread 0xffffdbffc6c0 (LWP 431162) exited]
[New Thread 0xffffdbffc6c0 (LWP 431191)]
[Thread 0xffffdbffc6c0 (LWP 431191) exited]
[New Thread 0xffffdbffc6c0 (LWP 431486)]
[Thread 0xffffdbffc6c0 (LWP 431486) exited]

Thread 5 "Server" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xffffedbfa6c0 (LWP 431158)]
0x0000fffff7cc3e0c in ?? () from /lib/aarch64-linux-gnu/libluajit-5.1.so.2
#0  0x0000fffff7cc3e0c in ?? () from /lib/aarch64-linux-gnu/libluajit-5.1.so.2
#1  0x0000fffff7d132b0 in lua_pcall ()
   from /lib/aarch64-linux-gnu/libluajit-5.1.so.2
#2  0x00000002001258c8 in ScriptApiBase::runCallbacksRaw(int, RunCallbacksMode, char const*) ()
#3  0x00000002001293f4 in ScriptApiEnv::environment_Step(float) ()
#4  0x0000000200392af8 in ServerEnvironment::step(float) ()
#5  0x0000000200378e0c in Server::AsyncRunStep(bool) ()
#6  0x000000020037b210 in ServerThread::run() ()
#7  0x00000002001aeeb8 in Thread::threadProc(Thread*) ()
#8  0x0000fffff7966cac in ?? () from /lib/aarch64-linux-gnu/libstdc++.so.6
#9  0x0000fffff7c06648 in start_thread (arg=0xffffedbf9fc0)
    at pthread_create.c:477
#10 0x0000fffff7729c1c in thread_start ()
    at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

[sfan5]

Try recompiling Minetest with LuaJIT from git first to see if that fixes the crashes.

[gwech]

@sfan5
Thanks for the advice.
So, I removed the already installed LuaJIT and reinstalled it from git.
Sadly, the minetest server still crashes with a segmentation fault :/

Let me know if you have other idea I could try.

@cyberarm
Did this worked for you?

[gwech]

@sfan5
Today I did something a bit redical but seems to have worked.

I totally wipped the Raspberry server and did a full reinstall.
I replaced the preinstalled LuaJIT package, and installed it from git.
After this I reinstalled minetest from stable-5 and did a build with the same cmake selection as I was trying before.

Right now the server looks stable, I had no Segentation Fault.
I now have added 20+ mods and everything looks good.

So, I feel that replacing the LuaJIT package does fix the issue, but I had to do this before the first minetest compile.
Instead of wipping the Raspberry OS, I probably should have deleted the minetest folder and restart from there.
I will keep monitoring and stress test the server as much as possible.

Thanks @sfan5 for suggesting the LuaJIT package replacement, it looks like it fixed the problem.
FYI @Zughy

[gwech]

@sfan5 @Zughy
Sorry, it actually crashed again :/
It is only working when compiled without LuaJIT ;(

The problem is still there!

So, any advice is still welcome to get this resolved.
Thanks

[Zughy]

Uhm, if the problem is LuaJIT itself, it's not up to us, is it?

[cyberarm]

@cyberarm
Did this worked for you?

It seems to be working. It's been running since a bit before sfan5 posted.

I ensured that the luajit-dev package was completely removed and then cloned luajit from https://luajit.org/git/luajit.git.
A little make PREFIX=/usr and sudo make install PREFIX=/usr to install luajit from source and nuking minetest's cmake build files before re-compiling it seems to have fixed the crashes.

(The server was crashing less and less often each time I restarted the server before trying without luajit and then running with luajit from source so it might still crash- eventually.)

https://luajit.org/install.html

[gwech]

Thanks @cyberarm
I did not remove the installed LuaJIT properly in the first place, I think!

Anyway, reinstalling LuaJIT from the git repositary seem to have fixed the issue.

Thank you all for you help
@sfan5
@Zughy