Architecture V5

[sgammon]

We need to make some changes to the way Elide is structured; the monorepo is hitting a point where it is heavy to work with, which is slowing velocity, both in mechanical circumstances (i.e. CI, releases...) and during development.

This new architecture aims to keep most of what was introduced in v4, which was a major departure from the Bazel-oriented structure in v3. There should be a specific suite of goals which land at roughly the same endpoint, over time, rather than a complete restructuring of what Elide is and how it is expressed as software.

Goals for v5

What follows is a suite of goals and anti-goals for this proposed new architecture:

• Goal: Cheaper builds
• Goal: Better integration with native code
• Goal: Smaller binary size
• Goal: Safer native code
• Anti-goal: Complete solution for binary size problem
• Anti-goal: Complete re-architecture of Elide at a conceptual level (like before lol)

Conceptual Changes

Elide has grown and solidified as an idea. The framework aspect of Elide's use hasn't really caught on, but the runtime idea has; we spend a lot of energy maintaining framework-style use, which is probably superfluous now. In this new architecture, we should focus less on the concept of a JVM framework and more on the final endpoint of a runtime which is useful as a developer tool, shipped as a native binary.

Build-time DI

If our final target is mostly native use, we probably don't need a full-blown JSR-compliant DI container in our own targets. We can potentially get creative here and use build-time reflection technologies (things like Reflekt) instead of traditional runtime reflection.

Reflective capabilities aren't used much host-side, and guest-side those capabilities are entirely separate. Build-time reflection improvements are likely to yield downstream improvements in binary size and performance as compiler visibility increases.

• Investigate build-time DI like Lightsaber, Kotlin Inject, and Koin. Prioritize:
  • Full support for all KMP targets (especially WASI)
  • Build-time/compile-time capabilities
  • General overhead and DX

Build-time JIT

Truffle is fully available at build-time, and we haven't done much tuning with regard to the JIT's use for built-in code. We could probably do something smarter here like embedded tests or small programs which exercise the runtime at build time, allowing for JIT to warm up for guest languages before it is snapshotted and held for runtime use.

Like the DI changes proposed above, these changes are likely to increase compiler understanding of the runtime, and so could yield binary size and performance improvements.

• Investigate better reporting of data from Points-to and Ideal Graph analysis
  • For example, Seafoam

Static JNI

Unpacking and loading native libraries at runtime is slow, necessarily, because it involves several steps which are individually slow: (1) seeking through the binary to find (typically large) embedded resources, (2) unpacking and de-compressing those resources to a place on disk which is safe for use, and (3) engaging in the JVM dance required to load a native library from either an absolute path or java.library.path, both of which are annoying for their own reasons.

Linking these libraries at build time instead improves binary size (because such data is no longer embedded, but inlined as code), and performance (because such code is loaded as part of the binary). Because this code is also now visible to the native compiler, and to GraalVM, optimization can take place on top of it.

We don't load a ton of native libraries at runtime. Most of them are from Netty and are well known to the team already. Thus, embedding these libs statically makes sense from a devex perspective too.

• Inventory additional libraries we need to convert, embed, or otherwise link against
• Currently known include:
  • Netty native transports
    • epoll
    • kqueue
    • unix
  • Netty native crypto (BoringSSL)
  • Netty native DNS (macOS only)
  • Netty HTTP3 (Quiche)
  • Native terminal libraries
    • Jansi
    • JLine
  • General native libraries
    • JNA (Needed for OSHI, etc.)

Rustification

Along with our static JNI stuff, we should endeavor to write as much of our native layer in Rust. Where possible, we can re-implement native libraries as we go, or add new functionality with JNI-based interfaces, on top of static linkage through org.graalvm..Feature implementations. These native binaries can be distributed easily in JARs using our new build-time unpacking logic.

We already have Rust embedded in our binary, at the hidden elide lint command, which is in early development for use with the new tooling API. We could extend this to things like native transports, where memory safety is particularly important, and where performance gain is particularly effective against top-line metrics.

Dynamic Plugins

For several reasons, dynamic language plugins (and tooling plugins) are growing more attractive (primarily, binary size). It doesn't seem possible yet to split our binary into shared libraries for each language, but research is ongoing, and the GraalVM team seems welcome to ideas in this area.

• What is the landscape of frameworks for native plugins?
• How do we solve the Truffle question?
• What about resources? What about reflection?

Reusable Tooling

To build on the success of our amazing Gradle build conventions, we should publish our tooling in binary form to an internal repository, which we can use as a means to split the project into smaller repositories. This will enable us to move the Kotlin Multiplatform side of our build to specific repositories which need it, and then downstream repos can consume these libs (and the associated conventions) without needing to rebuild them, or needing to index and keep modules in-memory, both of which are growing costly.

• Use our new build-infra repo for Elide itself
• Ship our conventions in an easily accessible Maven repo, so we can use them in other builds
• Continue work toward a unified native build-and-ship substrate (Zig + Cosmo)

Other R&D

There are tools for working with binaries. We've tried compression before and it hasn't yielded great results; tools like upx can mangle binaries or make them even slower. So, what do we do here? What does it look like to think outside the box? Are there other tools we should be trying? And so on.

• What does it look like to ship to other targets, like LLVM bitcode?
• Continued R&D edge work on Cosmo

[darvld]

Adding to the general goals and roadmap, here are some notes and opinions on current/future implementation details and features.

Lessons learned

Some lessons learned from the current architecture and the ongoing experiments. Most of the issues listed stem from features implemented by yours truly, with the best intentions at the time, but which slowly became obsolete as our vision of the project evolved; now we can reevaluate them and remove/rebuild/adjust as needed.

Opt-In markers

The @DelicateElideApi annotation is now obsolete, it was intended to prevent framework users from accidentally breaking the runtime by accessing internal APIs, but that is no longer a concern. It has become a hindrance as the vast majority of symbols require the annotation now (or an opt-in).

Opt-in annotations are one of my favorite Kotlin features, and we should use them, but only where it actually makes sense. For example, an @ElideInternal annotation can be used to guard access to the GraalVM components exposed by the runtime wrappers, as they are not meant to be accessed too often and can potentially cause issues if not used carefully.

Runtime abstractions

The abstractions currently in place over GraalVM components (PolyglotEngine and PolyglotContext) are useful, but they can be streamlined to allow more convenient access to the underlying Engine and Context instances. When they were originally designed, the goal was to avoid coupling the framework users' code to the GraalVM API, which is no longer a concern.

A change in the naming convention would also help readability, since the current names, "Polylot Engine" and "Polyglot Context", can be easily confused with the GraalVM components they wrap, but we will cross that bridge when we get to it.

Core APIs

We should strive for a language-agnostic runtime core module, to prevent an excessive bias toward JavaScript (like we have now) that may hinder future efforts in building truly polyglot intrinsics and other features. Experimental work shows this is entirely possible and improves build and IDE performance, since it enables working on the runtime engine without the overhead of language-specific implementation details.

About the Plugins System DSL 3000™️

The current Plugins system (PolyglotEngine DSL) was originally designed so as to not impose a specific DI container on framework users, allowing imperative configuration (as opposed to declarative, DI-based configuration).

While the API served its purposed and was flexible enough to accommodate for some unforeseen use cases, it has become increasingly difficult to extend and is already causing severe issues for the implementation of some features.

The next Extensions API should be integrated with DI (using framework-agnostic annotations from javax.inject), and should provide easier access to the lower level components wrapped by the runtime abstractions.

CLI command handlers

The CLI module (and its more recent companion, runtime) handle most of the "final user" for the runtime. As the number of features offered by Elide increases, classes like the ToolShellCommand handler grow excessively large, as they contain all the logic necessary to process user input and evaluate guest code from it.

A future architecture should split these responsibilities into (at least) two layers: user input (configuration files, CLI arguments) processing, and guest code execution. Extracting the evaluation logic into specialized service interfaces will provide us with more room for testing these features in isolation and drastically reduce the complexity of the command handlers.

Complex intrinsics

Certain features provided by the runtime can be quite complex in their logic, and draw dependencies that require special attention, such as the HTTP serving intrinsics. The current implementation (using Netty) requires us to manage the packaging of native libraries and apply many fine-tuning options to achieve a good end result.

These features are complex enough to warrant their own module, as they are not typically required, say, when working on the runtime core, or when fixing a bug in the JavaScript support logic. Extracting them into a separate project (not repository, simply a sub-project in the Elide build) could help the development experience significantly.

Elide Next-Gen

An experimental project called elide-nextgen is currently under development, and has already yielded interesting results that we may want to incorporate over time as we move toward the new architecture.

New abstractions

Rather than use PolyglotEngine and PolyglotContext to hide GraalVM APIs, we can use the wrapper only to enhance them. Elide is built on GraalVM at its core and it is unlikely that this will change in the foreseeable future, so it makes sense to have better integration with it from a design perspective:

fun sample() {
  // use default configuration values
  val config = RuntimeConfig()

  // all-in-one extension + language provided by `elide-js`
  val javascript = JavaScriptExtension()

  // prepare the extensions and languages
  val extensions = setOf(javascript)
  val languages = setOf(javascript)

  // create a new runtime instance, a wrapper over GraalVM Engine
  // that adds support for extensions and other high-level features
  val runtime = GuestRuntime.create(config, extensions, languages)

  // a scope wraps a GraalVM Context
  val scope = runtime.newScope()
  scope.context.eval("js", "console.log('hello')")
}

These abstractions can easily be integrated with a DI container, for example Micronaut:

@Factory class RuntimeFactory {
    @Singleton fun provideRuntimeConfig(
        // create configuration using Micronaut's
        // configuration properties
        properties: RuntimeConfigurationProperties
    ): RuntimeConfig {
        // map properties to the runtime config struct
        ...
    }

    @Singleton fun provideRuntime(
        config: RuntimeConfig,
        // DI discovers all available implementations
        extensions: List<RuntimeExtension>,
        languages: List<RuntimeLanguage>,
    ): GuestRuntime {
        return GuestRuntime.create(config, extensions, languages)
    }
}

The idea behind these abstractions is that they only enhance the wrapped components: GuestRuntime still provides access to its engine and GuestScope allows using its context when necessary, but they add features like extensions and context-bound metadata.

Enhanced extensions

An improvement over the current Plugins API, this extension system is meant to be used with DI: extensions are classes implementing RuntimeExtension, and receive event callbacks of their choice:

class MyExtension : RuntimeExtension {
  // configure a GraalVM engine builder before it is wrapped by a GuestRuntime
  override fun configureEngine(builder: Engine.Builder) {
    // set a boolean option to "true" (a string value is used)
    enableOption("engine.BackgroundCompilation")
  }

  // run initializer code on a GuestScope after its context is built, this is
  // where bindings are installed, init scripts are run, etc.
  override fun initializeScope(scope: GuestScope) {
    scope.context.eval("js", "console.log('initialized')")
  }

  // run cleanup code after the `initializeScope` event has been called for all
  // extensions, this allows more complex interactions between extensions
  override fun finalizeScope(scope: GuestScope) {
    // remove a binding at the end of the initialization phase
    scope.context.getBindings("js").remove("myInternalBinding")
  }
}

Additional events are available to configure the Context builder used for GuestScope instances, as well as initialize/finalize GuestRuntime instances.

Execution order

Another long-standing problem with the Plugins API is that there is no guarantee over the order in which plugins are called, and since they also don't have access to a DI container, it is very difficult to establish effective dependencies between plugins or with other components.

The new Extensions system addresses this problem directly, and allows extensions to depend on each other, establishing the order in which they are called by the runtime:

class MyExtension : RuntimeExtension {
    override fun install(handle: RuntimeExtension.Handle) {
        // this extensions needs bindings to be installed first,
        // guest bindings will be present in the scope during init
        handle.runAfter(BINDINGS_EXTENSION)

        // ensure we run before the embedded guest scripts, e.g.
        // to prepare some internal state for them
        handle.runBefore(GUEST_SCRIPTS_EXTENSION)
    }

    override fun initializeScope(scope: GuestScope) {
        // we can run code that uses bindings added by the
        // bindings extension, the order is respected
    }
}

Dependency loops between extensions are not allowed and will throw an
exception.

Specialized language extensions

Adding support for a guest language is done through the RuntimeLanguage API; implementations may choose to enable the language for a given Engine or Context. Typically a RuntimeLanguage will also implement RuntimeExtension for convenience, but this is not required.

class JavaScriptLanguage : RuntimeLanguage {
    override val languageId: GuestLanguage = "js"

    // choose whether to be enabled in a given engine; since the language
    // list must be passed to the engine builder on construction, it is 
    // currently not possible to access the engine itself during this check
    override fun enabledInEngine() {
        // always enable JavaScript (default implementation)
        return true
    }
}

Improved support for Guest Bindings

Bindings were never fully integrated with DI in the current architecture, because of how the DSL requires them to be registered. Additionally, language plugins needed to opt-in during context initialization in a way which was prone to error and hard to reason about.

A new approach to bindings now allows for much better control over which symbols are present in a guest context, and provides many useful features such as binding lifecycles.

Dedicated bindings registration API

A new interface, the GuestBindingRegistrar can be implemented by components that wish to inject values into a guest context. The registrar can add an arbitrary number of bindings, and can select a lifecycle and type for each one:

class MyBindingsRegistrar : GuestBindingRegistrar {
    override fun register(bindings: GuestBindingRegistry, scope: GuestScope) {
        // context-bound values are always available
        bindings.register("message", "hello", LANGUAGE_JS, GuestBindingLifecycle.CONTEXT)

        // init-bound values are removed after scope initialization
        bindings.register("limited", "temp", LANGUAGE_JS, GuestBindingLifecycle.INITIALIZER)
    }
}

Binding lifecycle

Some Guest Bindings are meant to be available always, acting as language or platform intrinsics (e.g. console in JavaScript). However, some bindings are not meant to be used by end-user code, and are only accessed by initialization scripts. The GuestBindingLifecycle API allows specifying how long a binding should be visible in a context, to avoid leaking it to runtime users:

• CONTEXT allows a binding to be accessed during the entire life of the context; useful for language intrinsics and other long-lived symbols.
• INITIALIZER removes the symbol after the context is initialized (during the finalizeScope event); suitable for bindings used by init scripts.

JPMS plugin

As part of the efforts to improve the build experience, the JPMS convention was extracted to a separate plugin, with its own project extension for configuring some settings, and added task descriptions and grouping.

Overall the plugin is much more stable than the convention and works as expected in the most common use cases, making JPMS support painless.

Buildkit plugin

A subtle but significant improvement to the build, the Buildkit plugin exposes the Options API, which reads Gradle properties from all available sources and provides type safety when accessing their values:

// build.gradle.kts

// searches for the 'elide.cool.enabled' Gradle property, and reads its
// value as a Boolean; returns false if not found.
val useCoolFeature: Boolean = optionEnabled("cool.enabled")

// searches for the 'elide.cool.name' property, failing if not found
val coolName: String = option("cool.name")

// same as the above but using a default value instead of failing
val coolButWithDefault: String = option("cool.name", default = "Elide")

Synthetic Proxies plugin (🚧 under development 🚧)

Bridging guest and host code is best handled by using the Proxy interfaces, but these are tedious to implement by hand and take up a lot of space. The goal is to implement a Kotlin Compiler plugin that adds a synthetic Proxy interface implementations to compatible classes with certain annotations, and generates the required methods:

// this class will implement `ProxyObject` at compile time; only properties
// and methods annotated with @ProxyMember will be exposed as members
@SyntheticProxy class MyIntrinsic {
    // properties can be registered as members, read-only properties
    // will remain so when the proxy is implemented
    @get:ProxyMember val message: String = "Hello!"
    
    // methods are added as executable members
    @ProxyMember fun hello() {
        println(message)
    }
}

[sgammon]

Yep, you already know I 100% agree with all of this.