Please support Spring Native hints to make it easy to turn your Exposed + Spring Boot application into GraalVM images

[joshlong]

Hi,

I am a big fan of Exposed. Thank you for the work you've done.

We on the Spring team are working to make it easier to support native applications with GraalVM. I use Exposed a lot for my talks so I wanted to make sure that it too would work with GraalVM native images, so I built a set of Spring Native "hints." Hints are defined as annotations and callback interfaces that generate configuration that gets fed into the GraalVM compiler to make your application work as a GraalVm native image.

If you go to the Spring Initialzr, and choose Spring Native (Experimental), it'll automatically configure a build that will produce a native image for you if you run mvn -Pnative package. The same is true for Gradle.

It even supports all the major usecases of Kotlin + Spring Boot users. Well, almost all of them. I'd like for there to be Spring Native hints for Exposed.

We can't put everything in the Spring Native project. The hints are best maintained by the projects that need them. So, I put together some hints for Exposed, and I was hoping to donate them to you. You can see there's not much in the way of code - it's all annotations in this particular case, though it could become more. I confess I don't know if this will support all the Exposed cornercases. But it does work for the simple demonstration application here.

I'd love to donate that hints class (and the supporting src/main/resources/META-INF/services/* files) to your amazing project so that people can simply add your hints library to the spring-aot plugins's dependencies and benefit from it.

There are a few wrinkles:

Spring Native is not yet GA.

Also, I don't really know much about Gradle, so my project is all using Maven. I am hoping somebody from your team would help me add it to your project

Thanks again for your wonderful work and I hope you're all doing well

[Tapac]

Hi @joshlong !
It's a very plesant to hear such kind words from Spring team member.
Sorry for the delayed answer - the issue comes out of my sight somehow.

I can prepare a separate module in the branch and put TypeHints from your repo there but can you advice me how to setup a test for that? Should I build some native image and run it or what?
If you can point me to documentation or sample/another project where similar thing was made it would help much.

[joshlong]

Hi there, so many years later, how are you? in the intervening years, we have moved on from Spring Native, which was experimental, to the Spring AOT component model, which has been part of Spring Boot 3.x since november 2022 as a GA technology, so I'd like to ask you to please consider supporting it instead of Spring Native.

GraalVm offers very exciting possibilities.

I've prototyped some basic support, and here's the example code - both a demo and the AOT code.

package com.example.exposed

import org.jetbrains.exposed.dao.id.IntIdTable
import org.jetbrains.exposed.sql.*
import org.jetbrains.exposed.sql.statements.api.PreparedStatementApi
import org.jetbrains.exposed.sql.transactions.TransactionManager
import org.springframework.aot.hint.MemberCategory
import org.springframework.aot.hint.RuntimeHints
import org.springframework.aot.hint.RuntimeHintsRegistrar
import org.springframework.boot.ApplicationArguments
import org.springframework.boot.ApplicationRunner
import org.springframework.boot.autoconfigure.SpringBootApplication
import org.springframework.boot.runApplication
import org.springframework.core.io.ClassPathResource
import org.springframework.stereotype.Component
import org.springframework.transaction.annotation.Transactional
import java.util.UUID.randomUUID

@ImportRuntimeHints(ExposedHints::class)
@SpringBootApplication
class ExposedApplication

fun main(args: Array<String>) {
    runApplication<ExposedApplication>(*args)
}

@Component
@Transactional
class Demo : ApplicationRunner {

    override fun run(args: ApplicationArguments?) {

        SchemaUtils.create(Customers, Orders)

        Orders.deleteAll()
        Customers.deleteAll()

        val ids: Iterable<Int> = listOf(
                "Olga", "Violetta", "Dr. Syer", "Stéphane", "Hadi", "Yuxin", "Josh", "Dave", "Madhura")
                .map { Customer(null, it) }
                .map { customer ->
                    Customers.insertAndGetId {
                        it[name] = customer.name
                    }
                }
                .map { it.value }

        val first = ids.first()
        listOf(randomUUID().toString(), randomUUID().toString())
                .forEach { sku ->
                    Orders.insert {
                        it[Orders.sku] = sku
                        it[Orders.customerId] = first
                    }
                }

        println("=".repeat(100))
        Customers
                .selectAll()
                .map { Customer(it[Customers.id].value, it[Customers.name]) }
                .forEach { println("got ${it}") }
        println("=".repeat(100))


        // query and print orders for the customer
       val ordersForCustomer =  (Customers innerJoin Orders)
               .selectAll().where { Orders.customerId eq first }
                .map { Order (it[Orders.id].value ,it[Orders.sku]) }
        println(ordersForCustomer)

    }
}

data class Customer(val id: Int?, val name: String)

data class Order(val id: Int, val sku: String)

object Orders : IntIdTable("orders") {
    val sku = text("sku")
    val customerId = reference("customerId", Customers) // This creates the "1 to many" relationship
}

object Customers : IntIdTable("customers") {
    val name = varchar("name", 50)
}


class ExposedHints : RuntimeHintsRegistrar {

    override fun registerHints(hints: RuntimeHints, classLoader: ClassLoader?) {

        arrayOf(
                org.jetbrains.exposed.spring.DatabaseInitializer::class,
                org.jetbrains.exposed.spring.SpringTransactionManager::class,
                java.util.Collections::class,
                Column::class,
                Database::class,
                Op::class ,
                Op.Companion::class ,
                DdlAware::class,
                Expression::class,
                ExpressionWithColumnType::class,
                ColumnType::class,
                DatabaseConfig::class,
                IColumnType::class,
                IntegerColumnType::class,
                PreparedStatementApi::class,
                ForeignKeyConstraint::class,
                IColumnType::class,
                QueryBuilder::class,
                Table::class,
                Transaction::class,
                TransactionManager::class,
                Column::class,
                Database::class,
                kotlin.jvm.functions.Function0::class,
                kotlin.jvm.functions.Function1::class,
                kotlin.jvm.functions.Function2::class,
                kotlin.jvm.functions.Function3::class,
                kotlin.jvm.functions.Function4::class,
                kotlin.jvm.functions.Function5::class,
                kotlin.jvm.functions.Function6::class,
                kotlin.jvm.functions.Function7::class,
                kotlin.jvm.functions.Function8::class,
                kotlin.jvm.functions.Function9::class,
                kotlin.jvm.functions.Function10::class,
                kotlin.jvm.functions.Function11::class,
                kotlin.jvm.functions.Function12::class,
                kotlin.jvm.functions.Function13::class,
                kotlin.jvm.functions.Function14::class,
                kotlin.jvm.functions.Function15::class,
                kotlin.jvm.functions.Function16::class,
                kotlin.jvm.functions.Function17::class,
                kotlin.jvm.functions.Function18::class,
                kotlin.jvm.functions.Function19::class,
                kotlin.jvm.functions.Function20::class,
                kotlin.jvm.functions.Function21::class,
                kotlin.jvm.functions.Function22::class,
                kotlin.jvm.functions.FunctionN::class
            )
            .map {  it.java }
            .forEach {
                hints.reflection().registerType(it, *MemberCategory.values())
            }

        arrayOf("META-INF/services/org.jetbrains.exposed.dao.id.EntityIDFactory",
                "META-INF/services/org.jetbrains.exposed.sql.DatabaseConnectionAutoRegistration",
                "META-INF/services/org.jetbrains.exposed.sql.statements.GlobalStatementInterceptor")
                .map { ClassPathResource(it) }
                .forEach { hints.resources().registerResource(it) }
    }

}

the important part is the Hints class at the bottom. We register it with Spring Boot using the ImportRuntimeHints annotatino on the main class. You can add that annotation to your auto configuration classes and they'll pull in the aot code. Keep in mind that the Spring Boot 2 line doesn't support AOT, so if you want to support both lines, consider instead defining the hint in src/main/resources/META-INF/spring/aot.factories and then add a line, like this:

org.springframework.aot.hint.RuntimeHintsRegistrar=com.example.exposed.ExposedHints

[e5l]

Hey @joshlong, thanks for the report.

@bog-walk, could you please check?

[bog-walk]

Hi @joshlong Thanks very much for all your effort behind this feature request and for sharing a prototype.
In the process of adding runtime hints, we're running into some limitations when using the DAO approach, which relies quite a bit on reflection. Have you by any chance used the DAO classes with native images before?
Any suggestions on how to properly register or overcome the following KotlinReflectionInternalErrors would be greatly appreciated while we work on options.

---

The first error is thrown when creating any new entity instance due to these fields in EntityClass:

Stacktrace (collapsed): kotlin.reflect.jvm.internal.KotlinReflectionInternalError: Could not compute caller for function

kotlin.reflect.jvm.internal.KotlinReflectionInternalError: Could not compute caller for function: public constructor CustomerEntity(id: org.jetbrains.exposed.dao.id.EntityID<kotlin.Int>) defined in com.example.testerspringgraalvm.CustomerEntity[DeserializedClassConstructorDescriptor@39035ad4] (member = null)
        at kotlin.reflect.jvm.internal.KFunctionImpl$caller$2.invoke(KFunctionImpl.kt:98) ~[na:na]
        at kotlin.reflect.jvm.internal.KFunctionImpl$caller$2.invoke(KFunctionImpl.kt:64) ~[na:na]
        at kotlin.SafePublicationLazyImpl.getValue(LazyJVM.kt:107) ~[tester-spring-graalvm.exe:1.12.4]
        at kotlin.reflect.jvm.internal.KFunctionImpl.getCaller(KFunctionImpl.kt:64) ~[na:na]
        at kotlin.reflect.jvm.internal.KCallableImpl.call(KCallableImpl.kt:108) ~[tester-spring-graalvm.exe:1.12.4]
        at org.jetbrains.exposed.dao.EntityClass$entityCtor$1.invoke(EntityClass.kt:39) ~[na:na]
        at org.jetbrains.exposed.dao.EntityClass$entityCtor$1.invoke(EntityClass.kt:39) ~[na:na]
        at org.jetbrains.exposed.dao.EntityClass.createInstance(EntityClass.kt:338) ~[tester-spring-graalvm.exe:1.12.4]
        at org.jetbrains.exposed.dao.EntityClass.new(EntityClass.kt:377) ~[tester-spring-graalvm.exe:1.12.4]
        at org.jetbrains.exposed.dao.EntityClass.new(EntityClass.kt:360) ~[tester-spring-graalvm.exe:1.12.4]
        at com.example.testerspringgraalvm.Demo.run(TesterSpringGraalvmApplication.kt:87) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.lang.reflect.Method.invoke(Method.java:568) ~[tester-spring-graalvm.exe:na]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:351) ~[na:na]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765) ~[na:na]
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) ~[na:na]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:392) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765) ~[na:na]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:717) ~[na:na]
        at com.example.testerspringgraalvm.Demo$$SpringCGLIB$$0.run(<generated>) ~[tester-spring-graalvm.exe:na]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:777) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:767) ~[tester-spring-graalvm.exe:1.12.4]
        at java.base@17.0.10/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
        at java.base@17.0.10/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[na:na]
        at java.base@17.0.10/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[na:na]
        at java.base@17.0.10/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[tester-spring-graalvm.exe:na]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:765) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:330) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1342) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1331) ~[tester-spring-graalvm.exe:1.12.4]
        at com.example.testerspringgraalvm.TesterSpringGraalvmApplicationKt.main(TesterSpringGraalvmApplication.kt:213) ~[tester-spring-graalvm.exe:na]

Attempting a registration like the following doesn't resolve the issue, nor does making the fields public:

hints
    .reflection()
    .registerField(EntityClass::class.java.getDeclaredField("entityPrimaryCtor\$delegate"))

Entirely avoiding reflection by providing the function manually does fix it, but we'd prefer that DAO users not have to rely on this workaround if a better option exists:

class CustomerEntity(id: EntityID<Int>) : IntEntity(id) {
    var name by Customers.name
    val orders by OrderEntity referrersOn Orders.customerId
    companion object : IntEntityClass<CustomerEntity>(
        Customers,
        entityCtor = { CustomerEntity(it) }  // this explicit argument avoids reflection
    )
}

---

Another issue encountered when testing edge cases occurs when preloading relations, for example if one was trying to access all Orders for a collection of Customers:

CustomerEntity
    .all()
    .with(CustomerEntity::orders)
    .forEach {
        println("Customer ${it.name} with orders: ${it.orders.map { o -> o.sku }}")
    }

Stacktrace (collapsed): kotlin.reflect.jvm.internal.KotlinReflectionInternalError: No accessors or field is found for property val

kotlin.reflect.jvm.internal.KotlinReflectionInternalError: No accessors or field is found for property val com.example.testerspringgraalvm.CustomerEntity.orders: org.jetbrains.exposed.sql.SizedIterable<com.example.testerspringgraalvm.OrderEntity>
        at kotlin.reflect.jvm.internal.KPropertyImplKt.computeCallerForAccessor(KPropertyImpl.kt:281) ~[na:na]
        at kotlin.reflect.jvm.internal.KPropertyImplKt.access$computeCallerForAccessor(KPropertyImpl.kt:1) ~[na:na]
        at kotlin.reflect.jvm.internal.KPropertyImpl$Getter$caller$2.invoke(KPropertyImpl.kt:180) ~[na:na]
        at kotlin.reflect.jvm.internal.KPropertyImpl$Getter$caller$2.invoke(KPropertyImpl.kt:179) ~[na:na]
        at kotlin.SafePublicationLazyImpl.getValue(LazyJVM.kt:107) ~[tester-spring-graalvm.exe:1.12.4]
        at kotlin.reflect.jvm.internal.KPropertyImpl$Getter.getCaller(KPropertyImpl.kt:179) ~[tester-spring-graalvm.exe:1.12.4]
        at kotlin.reflect.jvm.ReflectJvmMapping.getJavaMethod(ReflectJvmMapping.kt:64) ~[na:na]
        at kotlin.reflect.jvm.ReflectJvmMapping.getJavaGetter(ReflectJvmMapping.kt:49) ~[na:na]
        at kotlin.reflect.jvm.KCallablesJvm.setAccessible(KCallablesJvm.kt:71) ~[na:na]
        at org.jetbrains.exposed.dao.ReferencesKt.getReferenceObjectFromDelegatedProperty(References.kt:168) ~[na:na]
        at org.jetbrains.exposed.dao.ReferencesKt.preloadRelations(References.kt:204) ~[na:na]
        at org.jetbrains.exposed.dao.ReferencesKt.preloadRelations$default(References.kt:181) ~[na:na]
        at org.jetbrains.exposed.dao.ReferencesKt.with(References.kt:300) ~[na:na]
        at com.example.testerspringgraalvm.Demo.run(TesterSpringGraalvmApplication.kt:135) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.lang.reflect.Method.invoke(Method.java:568) ~[tester-spring-graalvm.exe:na]
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:351) ~[na:na]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765) ~[na:na]
        at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) ~[na:na]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:392) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:765) ~[na:na]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:717) ~[na:na]
        at com.example.testerspringgraalvm.Demo$$SpringCGLIB$$0.run(<generated>) ~[tester-spring-graalvm.exe:na]
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:777) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:767) ~[tester-spring-graalvm.exe:1.12.4]
        at java.base@17.0.10/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[na:na]
        at java.base@17.0.10/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357) ~[na:na]
        at java.base@17.0.10/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[na:na]
        at java.base@17.0.10/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[tester-spring-graalvm.exe:na]
        at java.base@17.0.10/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596) ~[tester-spring-graalvm.exe:na]
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:765) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:330) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1342) ~[tester-spring-graalvm.exe:1.12.4]
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1331) ~[tester-spring-graalvm.exe:1.12.4]
        at com.example.testerspringgraalvm.TesterSpringGraalvmApplicationKt.main(TesterSpringGraalvmApplication.kt:215) ~[tester-spring-graalvm.exe:na]

Please let me know if you have any thoughts about how to configure the RuntimeHintsRegistrar implementation to better support DAO users.
Here's a link to the YouTrack issue EXPOSED-327 in the event you'd rather discuss more there.

[joshlong]

I don't know much about the Dao approach. We could do a zoom or something to work through the code together.. should be easy enough, I'd guess. Message me josh@joshlong.com and we can setup some time? I'm in Romanian time zone until Friday. We just need some way to discover and reflect on those types at compilation time in the body a RuntimeHintsRegistrar. If the types are Spring beans, it's even easier: we can register a BeanFactoryInitializationAotProcessor, which has access to all of the BeanDefinutions and their class definitions. So we can inspect those