Enso Analytics
Launching Enso programs instantly #10121
Replies: 5 comments
-
Eliminate JVM Startup OverheadEnso interpreter is written in Java using the Truffle framework and it takes up to dozen of seconds to get up to full speed. JVM is just in time compiled as well. As such the Enso interpreter is its first interpreted by the JVM too. Only then it gets compiled as regular Java code, but that is still not enough. We need to compile the Enso interpreter code via Truffle partial evaluation and that usually happens only after few seconds of executing the program. Native Image to the Rescue"Truffle typical" solution to the slow initialization of the interpreter is to use Native Image and compile the interpreter into native executable or library. The startup is then greatly improved - usually to few milliseconds. All such compiled interpreters - Graal.js, Graal Python - start instantly. The same trick works for Enso as well:
after necessary changes we can execute simple Enso program computing factorial in less then ProblemsThe #3638 has proven the Enso interpreter overhead caused by JVM can be avoided. However, there is another problem. Other languages, when calling into operating system, they usually call C libraries. Enso is different. Enso uses Java and calls to the JVM to make system calls! As such we have to mitigate the JVM cost also for Enso libraries! |
Beta Was this translation helpful? Give feedback.
-
Mitigate JVM overhead for (Standard) LibrariesEnso libraries are using Java as a way to deal with operating system. As such even a call to
As can be seen from the above issues, Java on Truffle a.k.a. Espresso was seen as the original solution to this problem - the Espresso solution has been prototyped and since #8641 it is even part of our continuous integration. However it hasn't really speed startup up. Still the Espresso solution may be needed for other reasons - especially cloud security and scalability of individual tenants inside of a single cloud VM. Native Image to the RescueAn alternative solution has been proposed by @4e6:
but everything seems to be working and parts of the But of course, there are problems... ProblemsThe set of compiled-in libraries is fixed at compilation of the native image. There is no way to load additional libraries and their Java classes dynamically. What shall happen when a 3rd party library needs to load additional classes? |
Beta Was this translation helpful? Give feedback.
-
3rd Party Library UsageIf we compile some Standard libraries into Enso launcher native executable, what will happen when a completely different library or another version of compiled-in library is used? What will happen when the user tries to invoke polyglot java import java.lang.System
main = System.exit 10e.g. when the user code tries to make calls into JVM by itself? Better LauncherThe simplest solution to this problem is to implement better Enso runner. It is common among launchers of Truffle languages to support
and let's make sure the new launcher supports switching between native and JVM mode:
Of course, such a double re-launching requires the launch to be fast - that's another topic. Using EspressoEspresso is a Truffle interpreter of JVM and can execute Java bytecode dynamically even inside of the native executable. Since #8641 the Espresso mode is tested as part of our CI and it is know to work to some extent. We shall keep Espresso working, but until its experimental status is changed, it is unlikely we'd put Espresso into production. Using IsolatesThe drawback of this solution is that it works only for Oracle GraalVM, not for GraalVM Community Edition as isolates aren't part of the open source edition. On the other hand, it can be a nice bonus for those who buy Enso Enterprise. The idea is to launch Enso interpreter as native library (e.g. "isolate), but use regular HotSpot JVM to load classes dynamically. Prototype works. Needs proper support in the launcher - e.g. #5298 |
Beta Was this translation helpful? Give feedback.
-
Speed Startup Really UpEliminating cost of JVM is one thing, but using the CPU properly, not doing needless computations, etc. is also important. For example there is an analysis that Import/Export takes more than a second on a native image based executable. That's awfully a lot. We need to improve our code and address these situations:
Ideally we should be able to execute a simple program with native image in less that few hundred milliseconds. |
Beta Was this translation helpful? Give feedback.
-
What's needed to productize native image Enso launcher?Majority of the tasks listed in this dicussion have already been finished. Time to come up with final a set of tasks to productize this native image based launcher. ReleaseThe release shall only contain the native image based launcher. Users can use
DevelopmentBuilding native image takes time and is a threat to edit/compile/test development cycle.
|
Beta Was this translation helpful? Give feedback.
-
Enso is dynamically compiled language built on top of JVM - e.g. GraalVM - and its Truffle framework as such programs written in Enso can run pretty fast - once the dynamic compiler does its job. This kind of peak performance is not a subject of this discussion.
However it may take some time until an Enso program gets up to full speed. Currently that time is longer than it should be. The subject of this discussion is to highlight, reference existing issues that are supposed to launch Enso instantly and put them into proper context ready for a discussion.
Beta Was this translation helpful? Give feedback.
All reactions