A way to disable tail call optimizations in LLVM for arm targets

[nikarh]

Hi, I am the maintainer of Tier 3 target armv7-sony-vita-newlibeabihf.

There is an issue with tail call optimization causing crashes on this specific target. As an example, this often happens in drop calls, which in System allocator do libc::free, or in thread code which calls destroy functions as a last call.

The exact cause of the issue is unknown and would be hard to figure out, it might be caused by llvm bug, an issue with opensource SDK used to create a binary for the target platform, or outright hardware limitation. What complicates the analysis most is the fact that there is essentially no way to debug code on this platform.

So far the best solution I found for this issue is outright disabling tail and sibling call optimizations in llvm for this platform. With it disabled, there are no crashes regardless of the opt-level. The problem is there is no flag in llvm to disable tail calls on arm.

What would be the best course of action to fix this problem? Should I make a PR to upstream llvm with a flag that allows disabling tail calls on arm, and then get it backported to rust llvm fork?

[nikarh]

After some help from wg-llvm in zulip I've discovered the cause of the crashes:

My rust code is compiled in Arm mode, but the newlib is compiled in thumb mode. When doing tail call optimization, in the result elf after linking the last instruction in the Arm function is b instead of bx, so no switching from Arm mode to Thumb mode happens causing a crash.

Most likely this is an issue with a linker used in vitasdk, and not related to llvm.
As a solution, I'll change the target spec for the target from arm to thumb, since all of the vitasdk libraries are compiled in thumb mode.
Also I'll try to fix the underlying issue with a linker.

I'm closing this issue, and if it ever turns out that it's caused by llvm and not vitasdk toolchain, I'll open a new one linking to this issue.