Reusing precompiled gems between different development builds of TruffleRuby

[eregon]

Reinstalling gems with native extensions can take a while, so we should ideally only change the ABI version (= RbConfig::CONFIG['ruby_version']) when needed.
I think we should probably update the ABI version manually (can be a simple number we increment) + have a CI check that if any of the headers, mkmf.rb or rbconfig.rb change then ensure the ABI version has been bumped.
That check is also similar to the Makefile check we have for "should we recompile core C extensions" in

truffleruby/src/main/c/Makefile

Line 27 in 3ac8cc4

BASIC_EXTCONF_DEPS := $(SPAWN_HELPER) $(TRUFFLE_POSIX) $(RUBY_HEADERS) $(RBCONFIG) $(MKMF)

and that seems to work reasonably well.

I think nowadays it's rather infrequent to change the ABI, we almost never touch headers.
For releases, the release version would be the ABI version. (maybe keeping an extra digit with the ABI version, but might be confusing)

We could also try to use a C ABI checking tool (https://lvc.github.io/abi-compliance-checker/) but I guess that's a lot more involved to setup and might be inaccurate.

[eregon]

FWIW I've tried https://github.com/lvc/abi-dumper which works kind of OK, but the output does not always have the right ordering and is platform-dependent. It also kept giving warnings. Also installing this tool might not always be easy.
The old mode https://github.com/lvc/abi-compliance-checker did not work at all for me, and comparing 2 abi-dumper dumps actually failed to notice a new function.

So let's do the manual review of ABI whenever one the files susceptible to change ABI is modified, and check that in CI by having a manually-updated "ABI version" and "check version" (which indicates it was reviewed and there is no ABI change).

[chrisseaton]

Can we have a debug flag right now to change the gem installation path and mix ABIs at our own risk?

[eregon]

I have a PR that implements the check in CI and only increasing the ABI version when needed, it should merge soon.

[eregon]

Fixed in 34c7477, now the ABI should only change when required.