LinkageError: loader (instance of sun/misc/Launcher$AppClassLoader): attempted duplicate class definition for name: "net/openhft/chronicle/queue/JDBCResultBinarylightMethodWriter"

[JanStureNielsen]

The Chronicle Queue JDBC test code:

try (JDBCService service = new JDBCService(in, out, () -> DriverManager.getConnection("jdbc:hsqldb:file:" + file.getAbsolutePath(), "SA", ""))) {

    JDBCStatement writer = service.createWriter();
    writer.executeUpdate("CREATE TABLE tableName (\n" +
                        "name VARCHAR(64) NOT NULL,\n" +
                        "num INT\n" +
                        ")\n");

    for (int i = 1; i < (long) noUpdates; i++)
        writer.executeUpdate("INSERT INTO tableName (name, num)\n" +
               "VALUES (?, ?)", "name", i);

        written = System.nanoTime() - start;
        AtomicLong queries = new AtomicLong();
        AtomicLong updates = new AtomicLong();
        CountingJDBCResult countingJDBCResult = new CountingJDBCResult(queries, updates);
        MethodReader methodReader = service.createReader(countingJDBCResult);
        while (updates.get() < noUpdates) {
            if (!methodReader.readOne())
                Thread.yield();
            }
            Closeable.closeQuietly(service);
        }
    }
}

appears to fail with:

java.lang.ClassNotFoundException: net.openhft.chronicle.queue.JDBCStatementBinarylightMethodWriterpackage net.openhft.chronicle.queue;
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:163)
	at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:76)
	at net.openhft.chronicle.wire.GenerateMethodWriter.createClass(GenerateMethodWriter.java:257)
	at net.openhft.chronicle.wire.GenerateMethodWriter.newClass(GenerateMethodWriter.java:101)
	at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.newClass(VanillaMethodWriterBuilder.java:202)
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
	at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.get(VanillaMethodWriterBuilder.java:180)
	at net.openhft.chronicle.queue.JDBCService.createWriter(JDBCService.java:82)
	at fm.jdbc.Application.run(Application.java:73)
	at fm.jdbc.Application.main(Application.java:45)
java.lang.AssertionError: java.lang.LinkageError: loader (instance of  sun/misc/Launcher$AppClassLoader): attempted  duplicate class definition for name: "net/openhft/chronicle/queue/JDBCResultBinarylightMethodWriter"
	at net.openhft.compiler.CompilerUtils.defineClass(CompilerUtils.java:164)
	at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:157)
	at net.openhft.compiler.CachedCompiler.loadFromJava(CachedCompiler.java:76)
	at net.openhft.chronicle.wire.GenerateMethodWriter.createClass(GenerateMethodWriter.java:257)
	at net.openhft.chronicle.wire.GenerateMethodWriter.newClass(GenerateMethodWriter.java:101)
	at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.newClass(VanillaMethodWriterBuilder.java:202)
	at java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1660)
	at net.openhft.chronicle.wire.VanillaMethodWriterBuilder.get(VanillaMethodWriterBuilder.java:180)
	at net.openhft.chronicle.queue.JDBCService.runLoop(JDBCService.java:58)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
	at net.openhft.chronicle.core.threads.CleaningThread.run(CleaningThread.java:38)
Caused by: java.lang.LinkageError: loader (instance of  sun/misc/Launcher$AppClassLoader): attempted  duplicate class definition for name: "net/openhft/chronicle/queue/JDBCResultBinarylightMethodWriter"
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:635)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at net.openhft.compiler.CompilerUtils.defineClass(CompilerUtils.java:159)
	... 12 more

starting with:

<groupId>net.openhft</groupId>
<artifactId>chronicle-bom</artifactId>
<version>2.19.291</version>

up to and including the latest version 2.19.360. Prior to that, version 2.19.290 appears to work.

[RobAustin]

I've applied the following code to the develop branch

} catch (Error e) {
            // this can occur due to a race condition, but rather than synchronizing its access we will ignore the duplicate atempt
            if (e.getMessage().contains("attempted duplicate class definition")) {
                try {
                    return Class.forName(packageName + '.' + className, true, classLoader);
                } catch (ClassNotFoundException e1) {
                    throw Jvm.rethrow(e1);
                }
            }
            throw Jvm.rethrow(new ClassNotFoundException(e.getMessage() + '\n' + imports, e));
  

in here

   private Class createClass() {

        JavaSouceCodeFormatter interfaceMethods = new JavaSouceCodeFormatter(1);
        JavaSouceCodeFormatter imports = new JavaSouceCodeFormatter();

        try {
            imports.append("package " + packageName + ";\n\n" +
                    "import net.openhft.chronicle.wire.*;\n" +
                    "import " + IntConversion.class.getName() + ";\n" +
                    "import " + LongConversion.class.getName() + ";\n" +
                    "import " + GenerateMethodWriter.class.getName() + ";\n" +
                    "import " + MessageHistory.class.getName() + ";\n" +
                    "import " + MethodReader.class.getName() + ";\n" +
                    "import " + UpdateInterceptor.class.getName() + ";\n" +
                    "import " + MethodId.class.getName() + ";\n" +
                    "import " + GenerateMethodWriter.class.getName() + ";\n" +
                    "import " + DocumentContext.class.getName() + ";\n" +
                    "import " + SharedDocumentContext.class.getName() + ";\n" +
                    "import " + MethodWriterInvocationHandlerSupplier.class.getName() + ";\n" +
                    "import " + Jvm.class.getName() + ";\n" +
                    "import " + Closeable.class.getName() + ";\n" +
                    "import " + DocumentContextHolder.class.getName() + ";\n" +
                    "import " + MethodWriterListener.class.getName() + ";\n" +
                    "import java.lang.reflect.InvocationHandler;\n" +
                    "import java.lang.reflect.Method;\n" +
                    "import java.util.stream.IntStream;\n" +
                    "import java.util.ArrayList;\n" +
                    "import java.util.List;\n\n");

            imports.append("public final class ")
                    .append(className)
                    .append(" implements ");

            for (Class interfaceClazz : interfaces) {

                String interfaceName = nameForClass(interfaceClazz);
                imports.append(interfaceName);
                imports.append(",");

                if (!interfaceClazz.isInterface())
                    throw new IllegalArgumentException("expecting and interface instead of class=" + interfaceClazz.getName());

                for (Method dm : interfaceClazz.getMethods()) {
                    if (dm.isDefault() || Modifier.isStatic(dm.getModifiers()))
                        continue;

                    if (dm.getName().equals("documentContext")
                            && dm.getParameterTypes().length == 1
                            && dm.getParameterTypes()[0] == ThreadLocal.class
                            && SharedDocumentContext.class.isAssignableFrom(dm.getReturnType()))
                        continue;

                    interfaceMethods.append(createMethod(dm, interfaceClazz));
                }

            }

            imports.setLength(imports.length() - 1);
            imports.append("{\n\n");
            imports.append(constructorAndFields(className));
            imports.append(methodDocumentContext());
            imports.append(interfaceMethods);
            imports.append("\n}\n");

            if (DUMP_CODE)
                System.out.println(imports);

            return CACHED_COMPILER.loadFromJava(classLoader, packageName + '.' + className, imports.toString());

        } catch (Error e) {
            // this can occur due to a race condition, but rather than synchronizing its access we will ignore the duplicate atempt
            if (e.getMessage().contains("attempted duplicate class definition")) {
                try {
                    return Class.forName(packageName + '.' + className, true, classLoader);
                } catch (ClassNotFoundException e1) {
                    throw Jvm.rethrow(e1);
                }
            }
            throw Jvm.rethrow(new ClassNotFoundException(e.getMessage() + '\n' + imports, e));
        } catch (Throwable e) {
            throw Jvm.rethrow(new ClassNotFoundException(e.getMessage() + '\n' + imports, e));
        }
    }

@JanStureNielsen I can't release this branch at the moment, but if you could review this change and let me know if it fixes it for you, regards.

[JanStureNielsen]

Thanks for looking at this so quickly @RobAustin! Curiously, the race-to-define always fails for JDBCStatementBinarylightMethodWriter -- why is it always that class? I've made two additional comments in 85fb5ba.

[RobAustin]

As I’m sure you’re already aware we use code generation, and you have a use case where it is attempting to create the same class into different threads and at the same time ( and into the the same class loader). The code that Is doing the code creation checks to see if the class already exist, and if it does not exist it then goes ahead and creates it. We could make it thread safe and add synchronisation, but even if we did this, we Would have to be careful of the different class loaders. This may prevent the race but with the possibility of different class loaders it could be tricky to get correct and I was keen not to add an extra layer of blocking, and given it is a rare event I opted to instead catch the duplicate class and then just use the existing class. As this is somewhat simpler than synchronisation. I’m not sure this fully answered your question about why it is happening but all I can say is I saw it was happening and I think that I may have fixed it.

…

On 18 Jul 2020, at 14:12, Jan Nielsen ***@***.***> wrote:  Thanks for looking at this so quickly @RobAustin! Curiously, the race-to-define always fails for JDBCStatementBinarylightMethodWriter -- why is it always that class? I've made two additional comments in 85fb5ba. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[JanStureNielsen]

Closing as #177 is merged.

[hft-team-city]

Released in Chronicle-Wire-2.20.101, BOM-2.20.134