A cosmopolitan-native JVM/JDK

[jjsullivan5196]

Let me preface by saying, this is just an idea that I've had some interest in, and admittedly I lack experience in this area. If this isn't the right venue for this or it's just a dead end I totally understand.

Cosmo of course provides an experience familiar to that of the Java Virtual Machine (JVM) platform, but uses the amd64 ISA as its lingua franca to reach many platforms at once while being incredibly efficient. This also affords drastically simplifying toolchains, as compilers and all supporting programs can be shipped once as a single artifact, without the combinatorial explosion of platform/architecture/ISA woes.

While I am very impressed with the progress of porting so many languages and tools to cosmo, I still feel there's one affordance the JVM uniquely provides: in having the multitude of high-quality languages, libraries, protocol support, architectural patterns and more that can all be mixed and matched freely, on a coherent process VM that allows deep inspection and interaction at runtime. In today's world of many platforms, ways and places to run software however, there's an analysis paralysis that ensues when choosing the appropriate JVM and toolchain to work with. Graal has opened yet another can of worms trying to provide its own infrastructure for running high-level languages and runtimes close to the metal https://www.graalvm.org/latest/reference-manual/java-on-truffle/implementation/ And in purely historical terms, build tooling for the JVM and JDK has often been a churn-heavy responsibility that trickles down to those maintaining software repositories and OS platforms.

However now that research efforts have made it somewhat trivial to AOT compile Java, and JVM implementations written in Java, we might be closer to having a JVM that can predictably provide the capabilities of the host platform, all as a single software artifact. If we put that together with cosmo as the JVM's interface to the underlying platform, much of the headache of exposing such capabilities thru the JDK, JVM and its myriad native interfaces could be eliminated entirely. Given cosmo's library capabilities as well, there's probably a significant portion of the higher-level JDK APIs that could be shelled out to it.

To be clear, I'm not suggesting this is something that the cosmo community at large needs to care about or maintain, but I'd like to see what you folks out there think about this, given your experience using cosmo. I'm not aware of many people working on such a thing, but I have come across one example of an attempt already with an experimental JVM implementation http://bloggl.thenesis.org/2021/03/jainja-jvm-with-cosmopolitan-libc.html

[jjsullivan5196]

Should probably note, there is a feature request along these lines in the graal project, although it doesn't have too much activity right now oracle/graal#8350

[jart]

That all sounds great. Someone would need to will it into reality by building it though. As mentioned in oracle/graal#8350 Oracle themselves has done all the legwork supporting Musl, so since Cosmo is based on Musl, I can't imagine it'd be that big of a stretch to support Cosmo too. The biggest obstacle they're likely to encounter is getting JIT to work across OSes. Cosmo provides APIs like __jit_begin(), __jit_end(), and __clear_cache() to help with that. But if you look at libc/dlopen/dlopen.c you'll see it's not something that can be cleanly abstracted, if you care about OpenBSD and Apple M1.

[jjsullivan5196]

Yep, I'm starting by trying to get a build of espresso without JIT or a smaller JVM on cosmo. The jainja example is fully working, but it doesn't have an obvious path to full JDK 1.8 or higher support.

I'll ask for help here or on graal if I run into issues with native image, but wanted to start the discussion here incase someone was already deep in it.

[jart]

You and @sgammon are the two people who want this so far. @ahgamut uses Java, so he'd probably want what you're building enough to answer any questions you need to get unblocked on something.

[sgammon]

Hi @jjsullivan5196! I'd love to help any way I can. There is a healthy GraalVM community on Slack including an Espresso channel which has been quite helpful to me.

[sgammon]

Also, the GVM team often files internal tracking issues. Progress won't necessarily surface on GitHub. It may be worth a ping to the team to see if there has been work on this internally

[jjsullivan5196]

Thanks @sgammon ! I saw you touched that ticket again, I'll keep an eye on it incase anyone has news.

[jjsullivan5196]

Some notes on getting this to work so far @ahgamut

• GraalVM as a runtime gets lowered to native code thru its own compiler, it can be statically linked https://www.graalvm.org/22.0/reference-manual/native-image/StaticImages/
• There's a pretty comprehensive interface for linking against C headers in the compiler already (in Java), although I did run into some corners w/POSIX or GNU specific things not quite being in the right place [GR-52220] Support for Cosmopolitan Libc oracle/graal#8350 (comment)
• Main stumbling block at the moment is getting a binary seed JDK linked against cosmo. Graal's native-image compiler needs the native JDK interface static libs linked against the C library you're using before it'll let you use a new libc in native-image https://github.com/graalvm/labs-openjdk-22/blob/master/doc/building.md
• There's some constants and a couple APIs that may need to get changed in cosmo itself, there's a width in a signal API header I found that doesn't line up with POSIX (?) and sigevent isn't exported currently https://www.qnx.com/developers/docs/8.0/com.qnx.doc.neutrino.lib_ref/topic/s/sigevent.html

Some of that is reflected in these forks, though the real blocker right now is getting that labs JDK to build against cosmo.

https://github.com/jjsullivan5196/cosmopolitan/tree/graal-cosmo
https://github.com/jjsullivan5196/graal/tree/graal-cosmo

They also have this build tool called mx, @sgammon got me around some of it but how they put the different modules of graal together is still a little mysterious to me https://www.youtube.com/watch?v=3Gh0cz3vjG8

[jjsullivan5196]

@ahgamut

Is it possible to build GraalVM from source, using just GCC and no LLVM at compile-time or runtime? What build system does it use?

To answer this specifically, building GraalVM from source only needs GCC and a labs JDK, with the mx build system

What is the most minimal build of GraalVM that one can have? (can I build a GraalVM from source with only stdio, no networks/sockets)

And can't be sure about this one until the labs JDK can be built, the build system is pretty modular though, I know there are superfluous components of the Graal toolchain you can leave out

[jjsullivan5196]

Re the struct fields/API issues

https://pubs.opengroup.org/onlinepubs/009696699/basedefs/signal.h.html
jjsullivan5196@4c2c571
jjsullivan5196@8c5ea12