WIP: SN 0.5.6 -> 0.5.7 -> 0.5.8

[durban]

Notes so far:

• There is something weird going on with exception stacktraces; I haven't been able to minimize so far. UPDATE: seems to work fine in CI, so possibly something is broken in my local environment.
• ioAppTestsNative fails on macos-14 and ubuntu-22.04-arm

[durban]

Okay, I think I can turn on and off the (probably) segfaults with a scala-native config flag 😀

generateFunctionSourcePositions(true) // ioAppTestsNative fails in CI, and locally I get incorrect stacktraces
generateFunctionSourcePositions(false) // neither of those things happens

I can see, that stacktraces are computed in a different way if that flag is true (and this seems new in 0.5.7). My best guess is that that different way sometimes segfaults 🤷‍♂️ (and also causes incorrect stacktraces for me locally). Also, this seems related: scala-native/scala-native/pull/4094

[armanbilge]

Green CI, nice work!!

[durban]

Current status:

• generateFunctionSourcePositions(true) makes ioAppTestsNative fail on ubuntu-22.04-arm (I think, although lately I haven't seen it) and macos-14 (definitely). I believe both of those are ARM64. With false, everything is green.
• (true also causes incorrect exception stacktraces for me locally, but I also can't minimize this.)
• The CI failures are usually "segfault-like" (i.e., SIGSEGV or SIGBUS), although sometimes just exit code 1 (I'm not sure what's that about).
• -fcxx-exceptions / -fno-cxx-exceptions doesn't seem to make a difference either way.
• If I disable tracing in CE (i.e., hardwire stackTracingMode = "none"), everything is green, even with generateFunctionSourcePositions(true). (Of course, to do this, I had to disable a few tests which need tracing to work. This was just an experiment, I'll obviously rollback this.)

Based on these, I can't definitively prove, but I suspect this could be a scala-native issue. I can see, that stacktraces are calculated in a different way in 0.5.7 if generateFunctionSourcePositions is true. My best guess is that that different way sometimes segfaults 🤷‍♂️. (Also, this seems related: scala-native/scala-native#4094.)

[durban]

In 4e91a9c I've restored nativeconfig to what it was at the end of #4201, but I've originally added that for debugging segfaults. So maybe we can remove that entirely, and just use the SN default?

In any case, this is blocked (at least) until SN 0.5.8, unless of course we're fine with the workaround generateFunctionSourcePositions(false) ;-)

[keynmol]

Just to confirm (as I'm not seeing green CIs any more) – it's theoretically possible to build CE successfully on 0.5.7 with generateFunctionSourcePositions(false)?

If that is the case, IMO the best thing to do would be to get this out ASAP to start building the rest of the ecosystem, addressing the rest in a patch release (of either SN or CE)

[durban]

Just to confirm (as I'm not seeing green CIs any more) – it's theoretically possible to build CE successfully on 0.5.7 with generateFunctionSourcePositions(false)?

Yes. (To make sure, I've just pushed a899493, it should be green.)

[kubukoz]

shall this be un-WIPped, then? :)

Or maybe we can also get a 0.5.8, @WojciechMazur 🙏

[durban]

I see two ways to un-WIP this:

1. We definitively decide that the generateFunctionSourcePositions(false) workaround is fine. (I don't know enough about scala-native configurations to make this call.)
2. There is a 0.5.8, and it works fine.

[keynmol]

0.5.8 has been released just now

[durban]

This is "fun": with 0.5.8 there is a completely(?) different segfault. It's on x86_64 linux (at least I think "Native high-core-count" is x86_64).

[durban]

I think it's scala-native/scala-native/issues/4366 (more details there).

[djspiewak]

Okay so I think what we're going to do is release 3.7 (or at least, an RC) against 0.5.6, just to unblock the ecosystem and move the train forward. Excellent digging btw, @durban.

[durban]

Good news, everyone! It seems we can avoid scala-native/scala-native/issues/4366 by turning on the SN optimizer (see details there, tl;dr: alwaysinline isn't).

Apparently the optimizer being on is the default, and funnily we probably turned it off to make multithreading work easier. I don't know why would anyone want to turn it off (okay, I obviously know...), but we might just say in the release notes: "don't do that"?

[durban]

Also, I can now confirm that my fix for the SN 0.5.7 segfaults (scala-native/scala-native#4301) was indeed correct. (Due to the new segfaults in 0.5.8 that wasn't certain until now. But now I've tried (#4447), and 0.5.7 also segfaults with the optimizer on.)

[lolgab]

true is the default, so you can remove the line altogether.

Suggested change

.withOptimize(true)

[durban]

As long as stuff is broken with false, I'd prefer to be explicit. Also, to remove that line would mean removing the explanatory comment (you've reviewed an outdated commit).