XLST causes the executable to crash at runtime #3811

andirady · 2021-09-17T23:35:15Z

Describe the issue
The following XLST doesn't work at runtime after compiled with native-image but works in JVM.

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output omit-xml-declaration="yes" indent="yes"/>
    <xsl:strip-space elements="*"/>

    <xsl:template match="@*|node()">
        <xsl:copy>
            <xsl:apply-templates select="@*|node()"/>
        </xsl:copy>
    </xsl:template>

</xsl:stylesheet>

The problematic code portion is as follows:

Transformer transformer;
try (var xlst = XmlFormatter.class.getClassLoader().getResourceAsStream("formatter.xlst")) {
    transformer = tf.newTransformer(new StreamSource(xlst)); // crashed presumably happened here.
}

Steps to reproduce the issue
Please include both build steps as well as run steps

Download zip from https://gist.github.com/andirady/82a8407523a45453541a7f9ef2802462
Extract the zip
cd into the extracted directory
make xmlformatter
./xmlformatter test.xml

Describe GraalVM and your environment:

GraalVM CE 21.2.0 (build 16.0.2+7-jvmci-21.2-b08)
openjdk version "16.0.2" 2021-07-20
Windows Subsystem for Linux (Debian 4.4.0-19041-Microsoft)
Architecture: x86_64

More details

Fatal error reported via JNI: JVM_DefineModule called:  Unimplemented

JavaFrameAnchor dump:

  Anchor 00007ffff2c40758 LastJavaSP 00007ffff2c40730 LastJavaIP 00007f803bbe23e8
  
TopFrame info:

  TotalFrameSize in CodeInfoTable 80
  
VMThreads info:

  VMThread 00007fffea0cf280  STATUS_IN_JAVA (safepoints disabled)  java.lang.Thread@0x7f803a645380
  
VM Thread State for current thread 00007fffea0cf280:

  0 (8 bytes): com.oracle.svm.jni.JNIThreadLocalEnvironment.jniFunctions = (bytes) 
    00007fffea0cf280: 00007f803a3df010
  
  8 (8 bytes): com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.stackBoundaryTL = (Word) 140737257984000  00007ffff244d000
  16 (4 bytes): com.oracle.svm.core.thread.Safepoint.safepointRequested = (int) 2147032519  7ff91dc7
  20 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.statusTL = (int) 1  00000001
  24 (32 bytes): com.oracle.svm.core.genscavenge.ThreadLocalAllocation.regularTLAB = (bytes) 
    00007fffea0cf298: 00007f8037e00000 00007f8037f00000
    00007fffea0cf2a8: 00007f8037ed76c8 00007f803aec0000
    
  
  56 (8 bytes): com.oracle.svm.core.thread.JavaThreads.currentThread = (Object) java.lang.Thread  00007f803a645380
  64 (8 bytes): com.oracle.svm.core.stack.JavaFrameAnchors.lastAnchor = (Word) 140737266321240  00007ffff2c40758
  72 (8 bytes): com.oracle.svm.core.heap.NoAllocationVerifier.openVerifiers = (Object) null
  80 (8 bytes): com.oracle.svm.core.identityhashcode.IdentityHashCodeSupport.hashCodeGeneratorTL = (Object) java.util.SplittableRandom  00007f8039e40028
  88 (8 bytes): com.oracle.svm.core.snippets.ExceptionUnwind.currentException = (Object) null
  96 (8 bytes): com.oracle.svm.core.thread.ThreadingSupportImpl.activeTimer = (Object) null
  104 (8 bytes): com.oracle.svm.graal.isolated.IsolatedCompileClient.currentClient = (Object) null
  112 (8 bytes): com.oracle.svm.graal.isolated.IsolatedCompileContext.currentContext = (Object) null
  120 (8 bytes): com.oracle.svm.jni.JNIObjectHandles.handles = (Object) com.oracle.svm.core.handles.ThreadLocalHandles  00007f8039e01298
  128 (8 bytes): com.oracle.svm.jni.JNIThreadLocalPendingException.pendingException = (Object) null
  136 (8 bytes): com.oracle.svm.jni.JNIThreadLocalPinnedObjects.pinnedObjectsListHead = (Object) null
  144 (8 bytes): com.oracle.svm.jni.JNIThreadOwnedMonitors.ownedMonitors = (Object) null
  152 (8 bytes): com.oracle.svm.core.SubstrateDiagnostics.threadOnlyAttachedForCrashHandler = (bytes) 
    00007fffea0cf318: 0000000000000000
  
  160 (8 bytes): com.oracle.svm.core.thread.VMThreads.IsolateTL = (Word) 140188705619968  00007f803a000000
  168 (8 bytes): com.oracle.svm.core.thread.VMThreads.OSThreadHandleTL = (Word) 140188721680768  00007f803af51180
  176 (8 bytes): com.oracle.svm.core.thread.VMThreads.OSThreadIdTL = (Word) 140188721680768  00007f803af51180
  184 (8 bytes): com.oracle.svm.core.thread.VMThreads.nextTL = (Word) 0  0000000000000000
  192 (8 bytes): com.oracle.svm.core.thread.VMThreads.unalignedIsolateThreadMemoryTL = (Word) 140737120105088  00007fffea0cf280
  200 (4 bytes): com.oracle.svm.core.graal.snippets.StackOverflowCheckImpl.yellowZoneStateTL = (int) 1  00000001
  204 (4 bytes): com.oracle.svm.core.snippets.ImplicitExceptions.implicitExceptionsAreFatal = (int) 0  00000000
  208 (4 bytes): com.oracle.svm.core.thread.ThreadingSupportImpl.currentPauseDepth = (int) 0  00000000
  212 (4 bytes): com.oracle.svm.core.thread.VMThreads$ActionOnExitSafepointSupport.actionTL = (int) 0  00000000
  216 (4 bytes): com.oracle.svm.core.thread.VMThreads$ActionOnTransitionToJavaSupport.actionTL = (int) 0  00000000
  220 (4 bytes): com.oracle.svm.core.thread.VMThreads$StatusSupport.safepointsDisabledTL = (int) 1  00000001
  
VMOperation dump:

  No VMOperation in progress
  
  The 10 most recent VM operation status changes (oldest first):
    
  
Dump Counters:

  
Raw Stacktrace:

  00007ffff2c406d0: 0000000000000000 00007f803c223570
  00007ffff2c406e0: 00007fffea0cf280 00000000008e9688
  00007ffff2c406f0: 00007fffea0cf280 00007f803a000000
  00007ffff2c40700: 00007ffff2c40758 00007f803bbe23e8
  00007ffff2c40710: fffffffffded76b0 00007f803c21de44
  00007ffff2c40720: 00000000008e9688 00007f803bbe23fb
  00007ffff2c40730: 0000000000000004 00007f8037ed7660
  00007ffff2c40740: 00007f803a000000 00007f803a000000
  
Stacktrace Stage 0:

  SP 00007ffff2c406d0 IP 00007f803b4ac886  FrameSize 80
  SP 00007ffff2c40730 IP 00007f803bbe23e8  FrameSize 112
  SP 00007ffff2c407a0 IP 00007f803bbe0c0f  FrameSize 64
  SP 00007ffff2c407e0 IP 00007f803bbe2e6b  FrameSize 240
  SP 00007ffff2c408d0 IP 00007f803bbe8082  FrameSize 48
  SP 00007ffff2c40900 IP 00007f803bbe94ea  FrameSize 48
  SP 00007ffff2c40930 IP 00007f803bbe93bd  FrameSize 48
  SP 00007ffff2c40960 IP 00007f803b7e48d8  FrameSize 48
  SP 00007ffff2c40990 IP 00007f803b7e1564  FrameSize 16
  SP 00007ffff2c409a0 IP 00007f803bcbfebb  FrameSize 16
  SP 00007ffff2c409b0 IP 00007f803b7e2463  FrameSize 96
  SP 00007ffff2c40a10 IP 00007f803b7e2bb4  FrameSize 80
  SP 00007ffff2c40a60 IP 00007f803b7e42cc  FrameSize 64
  SP 00007ffff2c40aa0 IP 00007f803b7e4e66  FrameSize 48
  SP 00007ffff2c40ad0 IP 00007f803b7eb0f4  FrameSize 32
  SP 00007ffff2c40af0 IP 00007f803b498414  FrameSize 112
  SP 00007ffff2c40b60 IP 00007f803b4b21d6  FrameSize 112
  
Stacktrace Stage 1:

  SP 00007ffff2c406d0 IP 00007f803b4ac886  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40730 IP 00007f803bbe23e8  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c407a0 IP 00007f803bbe0c0f  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c407e0 IP 00007f803bbe2e6b  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c408d0 IP 00007f803bbe8082  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40900 IP 00007f803bbe94ea  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40930 IP 00007f803bbe93bd  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40960 IP 00007f803b7e48d8  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40990 IP 00007f803b7e1564  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c409a0 IP 00007f803bcbfebb  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c409b0 IP 00007f803b7e2463  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40a10 IP 00007f803b7e2bb4  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40a60 IP 00007f803b7e42cc  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40aa0 IP 00007f803b7e4e66  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40ad0 IP 00007f803b7eb0f4  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40af0 IP 00007f803b498414  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  SP 00007ffff2c40b60 IP 00007f803b4b21d6  com.oracle.svm.core.code.CodeInfo@0x7f803a5b36c0 name = image code
  
Stacktrace Stage 2:

  SP 00007ffff2c406d0 IP 00007f803b4ac886  [image code] com.oracle.svm.core.code.IsolateEnterStub.JNIFunctions_FatalError_dea4272cf8e20ff701a0a73f7a6d5d75e3401653(IsolateEnterStub.java:0)
  SP 00007ffff2c40730 IP 00007f803bbe23e8  [image code] java.lang.Module.defineModule0(Module.java)
  SP 00007ffff2c407a0 IP 00007f803bbe0c0f  [image code] java.lang.Module.<init>(Module.java:136)
  SP 00007ffff2c407e0 IP 00007f803bbe2e6b  [image code] java.lang.Module.defineModules(Module.java:1171)
  SP 00007ffff2c408d0 IP 00007f803bbe8082  [image code] java.lang.ModuleLayer.<init>(ModuleLayer.java:185)
  SP 00007ffff2c40900 IP 00007f803bbe94ea  [image code] java.lang.ModuleLayer.defineModules(ModuleLayer.java:670)
  SP 00007ffff2c40930 IP 00007f803bbe93bd  [image code] java.lang.ModuleLayer.defineModules(ModuleLayer.java:423)
  SP 00007ffff2c40960 IP 00007f803b7e48d8  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.lambda$createModule$1(TemplatesImpl.java:443)
  SP 00007ffff2c40990 IP 00007f803b7e1564  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl$$Lambda$5272a4cd586f731c9b47dd635fc35e6d54bed73b.run(Unknown Source)
  SP 00007ffff2c409a0 IP 00007f803bcbfebb  [image code] java.security.AccessController.doPrivileged(AccessController.java:87)
  SP 00007ffff2c409b0 IP 00007f803b7e2463  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.createModule(TemplatesImpl.java:444)
  SP 00007ffff2c40a10 IP 00007f803b7e2bb4  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.defineTransletClasses(TemplatesImpl.java:493)
  SP 00007ffff2c40a60 IP 00007f803b7e42cc  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.getTransletInstance(TemplatesImpl.java:551)
  SP 00007ffff2c40aa0 IP 00007f803b7e4e66  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.newTransformer(TemplatesImpl.java:584)
  SP 00007ffff2c40ad0 IP 00007f803b7eb0f4  [image code] com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTransformer(TransformerFactoryImpl.java:828)
  SP 00007ffff2c40af0 IP 00007f803b498414  [image code] XmlWriter.main(XmlWriter.java:23)
  SP 00007ffff2c40b60 IP 00007f803b4b21d6  [image code] com.oracle.svm.core.JavaMainWrapper.runCore(JavaMainWrapper.java:146)
  SP 00007ffff2c40b60 IP 00007f803b4b21d6  [image code] com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:182)
  SP 00007ffff2c40b60 IP 00007f803b4b21d6  [image code] com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(IsolateEnterStub.java:0)
  
[Heap settings and statistics: 
  Supports isolates: true
  Object reference size: 8
  Incremental collections: 0
  Complete collections: 0]

[Native image heap boundaries: 
  ReadOnly Primitives: 0x7f803a001028 .. 0x7f803a243a48
  ReadOnly References: 0x7f803a243a60 .. 0x7f803a3de488
  ReadOnly Relocatables: 0x7f803a3df000 .. 0x7f803a4f6e10
  Writable Primitives: 0x7f803a4f7000 .. 0x7f803a5eb7d0
  Writable References: 0x7f803a5eb7e0 .. 0x7f803a90b908
  Writable Huge: 0x0 .. 0x0
  ReadOnly Huge: 0x7f803aa00030 .. 0x7f803addb078]


[Heap:
  [Young generation: 
    [Eden: 
      [edenSpace:
        aligned: 0/0 unaligned: 0/0]]
    [Survivors: 
      ]]
  [Old generation: 
    [oldFromSpace:
      aligned: 0/0 unaligned: 0/0]
    [oldToSpace:
      aligned: 0/0 unaligned: 0/0]
    ]
  [Unused:
    aligned: 0/0]]

The text was updated successfully, but these errors were encountered:

rodrigar-mx · 2021-09-20T22:59:23Z

Hi @andirady. Thanks for reporting the issue. I will take a closer look into it and get back to you.

rodrigar-mx · 2021-09-21T19:09:40Z

@andirady, I have reported this issue with the native-image team. They will respond you accordingly.

vjovanov · 2021-09-22T04:41:57Z

This issue has been fixed but we are waiting for the backports to JDK 11 and JDK 17:

https://bugs.openjdk.java.net/browse/JDK-8273278

After this is merged, XSLT will be fully supported, however, it needs to use class pre-difinition for XSLT templates: one needs an agent run to collect all the generated template-processing classes.

jerboaa · 2021-10-05T17:43:24Z

First off, this appears to be an issue of xsltc which is the default TransformerFactory provider as part of OpenJDK. Note that there doesn't seem to be interpretive xalan available (as part of OpenJDK) and needs to be used via an external dependency. This is what this reproducer shows:

https://github.com/jerboaa/demo-xslt-native

Interpretive xalan works with native-image. xsltc based transforms don't because of a) non-determinism of generated bytecode of classes b) generated classes don't get picked up out of the box (even when run with the agent).

In my experiments the OpenJDK fix (JDK 17 or proposed JDK 11) alone doesn't work. For example a patched OpenJDK 11+graal vm build via mandrel results in:

$ ./xmlformatter test.xml 
xlst = java.io.ByteArrayInputStream@7f782e4d
Exception in thread "main" com.oracle.svm.core.jdk.UnsupportedFeatureError: No classes have been predefined during the image build to load from bytecodes at runtime.
	at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:87)
	at com.oracle.svm.core.hub.PredefinedClassesSupport.throwNoBytecodeClasses(PredefinedClassesSupport.java:76)
	at com.oracle.svm.core.hub.PredefinedClassesSupport.loadClass(PredefinedClassesSupport.java:130)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:269)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl$TransletClassLoader.defineClass(TemplatesImpl.java:207)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.defineTransletClasses(TemplatesImpl.java:514)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.getTransletInstance(TemplatesImpl.java:551)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.newTransformer(TemplatesImpl.java:584)
	at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTransformer(TransformerFactoryImpl.java:818)
	at XmlFormatter.main(XmlFormatter.java:23)
$ strings ./xmlformatter | grep com.oracle.svm.core.VM
com.oracle.svm.core.VM.Target.StaticLibraries=liblibchelper.a|libnet.a|libnio.a|libjava.a|libfdlibm.a|libsunec.a|libzip.a|libjvm.a
com.oracle.svm.core.VM.Java.Version=11.0.14-internal
com.oracle.svm.core.VM.Target.Platform=org.graalvm.nativeimage.Platform$LINUX_AMD64
com.oracle.svm.core.VM.Target.LibC=com.oracle.svm.core.posix.linux.libc.GLibC
com.oracle.svm.core.VM.Target.Libraries=stdc++|z|pthread|dl|rt
com.oracle.svm.core.VM=GraalVM 21.3.0-dev Java 11 Mandrel Distribution
com.oracle.svm.core.VM.Target.CCompiler=gcc|redhat|x86_64|11.2.1
$ cat cfg/predefined-classes-config.json 
[
  {
    "type":"agent-extracted",
    "classes":[
    ]
  }
]

This yields me to believe that some extra steps for telling graal vm where to find generated classes needs to happen. Thoughts?

jovanstevanovic · 2021-10-15T21:15:16Z

Hi @jerboaa, thank you for your thorough analysis!

I'm sending you a patch that will make things work. Try on your own, at let me know if everything is fine now. 😄

jerboaa · 2021-10-18T16:00:21Z

I'm sending you a patch that will make things work. Try on your own, at let me know if everything is fine now.

@jovanstevanovic Thanks. I've updated the demo with the patch. It "seems" to work for the unpatched xsltc reproducer (with upstream xalan). But that seems pure luck.

jovanstevanovic · 2021-10-20T13:20:27Z

It looks like the xsltc branch in the repo is running into a problem because of modules. So I will include @ivan-ristovic to verify that the problem is with the lack of NI module support.

ivan-ristovic · 2021-10-20T17:23:16Z

This seems to be related to our Unsafe.objectFieldOffset implementation being blocked by reflection filters. I will roll a patch that will address the issue and then the error mentioned here should be fixed.

jovanstevanovic · 2021-11-01T08:37:39Z

Hey @jerboaa, some good news! @ivan-ristovic's PR with this small patch makes it work. Try again, and let me know if something shows up. 😄

andirady added bug native-image labels Sep 17, 2021

rodrigar-mx self-assigned this Sep 20, 2021

rodrigar-mx assigned vjovanov and unassigned rodrigar-mx Sep 21, 2021

vjovanov assigned jovanstevanovic and unassigned vjovanov Sep 22, 2021

vjovanov added this to the 21.3 milestone Sep 22, 2021

This was referenced Oct 22, 2021

Substitute ClassLoader.createOrGetClassLoaderValueMap in order to make Module.defineModule0 work with XLST #3940

Closed

jdk.internal.misc.Unsafe.objectFieldOffset substitution is sensitive to reflection filters #3961

Open

fniephaus closed this as completed Sep 14, 2022

TharmiganK mentioned this issue Oct 13, 2022

Add native image build support for XSLT ballerina-platform/ballerina-library#3462

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

XLST causes the executable to crash at runtime #3811

XLST causes the executable to crash at runtime #3811

andirady commented Sep 17, 2021

rodrigar-mx commented Sep 20, 2021

rodrigar-mx commented Sep 21, 2021

vjovanov commented Sep 22, 2021 •

edited

jerboaa commented Oct 5, 2021

jovanstevanovic commented Oct 15, 2021

jerboaa commented Oct 18, 2021

jovanstevanovic commented Oct 20, 2021

ivan-ristovic commented Oct 20, 2021

jovanstevanovic commented Nov 1, 2021

XLST causes the executable to crash at runtime #3811

XLST causes the executable to crash at runtime #3811

Comments

andirady commented Sep 17, 2021

rodrigar-mx commented Sep 20, 2021

rodrigar-mx commented Sep 21, 2021

vjovanov commented Sep 22, 2021 • edited

jerboaa commented Oct 5, 2021

jovanstevanovic commented Oct 15, 2021

jerboaa commented Oct 18, 2021

jovanstevanovic commented Oct 20, 2021

ivan-ristovic commented Oct 20, 2021

jovanstevanovic commented Nov 1, 2021

vjovanov commented Sep 22, 2021 •

edited