Skip to content

[Bug] codegen error in Flink stream write job #6228

@hbgstc123

Description

@hbgstc123

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.9

Compute Engine

Flink 1.17

Minimal reproduce step

start a flink stram write job to write into a paimon table.

What doesn't meet your expectations?

throw error

taskName:Writer : paimon_upsert_bucket_stream_test01 taskId:683e1cff79211e54e7b4d15097c08c69
java.io.IOException: java.lang.RuntimeException: Could not instantiate generated class 'interface org.apache.paimon.codegen.RecordEqualiser'
	at org.apache.paimon.flink.sink.RowDataStoreWriteOperator.processElement(RowDataStoreWriteOperator.java:130)
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask$StreamTaskNetworkOutput.emitRecord(OneInputStreamTask.java:237)
	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.processElement(AbstractStreamTaskNetworkInput.java:146)
	at org.apache.flink.streaming.runtime.io.AbstractStreamTaskNetworkInput.emitNext(AbstractStreamTaskNetworkInput.java:110)
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:65)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:561)
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:231)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:886)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:835)
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:968)
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:947)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:761)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: Could not instantiate generated class 'interface org.apache.paimon.codegen.RecordEqualiser'
	at org.apache.paimon.codegen.CodeGenUtils.generate(CodeGenUtils.java:121)
	at org.apache.paimon.codegen.CodeGenUtils.newRecordEqualiser(CodeGenUtils.java:93)
	at org.apache.paimon.utils.ValueEqualiserSupplier.get(ValueEqualiserSupplier.java:43)
	at org.apache.paimon.utils.ValueEqualiserSupplier.get(ValueEqualiserSupplier.java:31)
	at org.apache.paimon.operation.KeyValueFileStoreWrite.createRewriter(KeyValueFileStoreWrite.java:328)
	at org.apache.paimon.operation.KeyValueFileStoreWrite.createCompactManager(KeyValueFileStoreWrite.java:245)
	at org.apache.paimon.operation.KeyValueFileStoreWrite.createWriter(KeyValueFileStoreWrite.java:206)
	at org.apache.paimon.operation.KeyValueFileStoreWrite.createWriter(KeyValueFileStoreWrite.java:94)
	at org.apache.paimon.operation.AbstractFileStoreWrite.createWriterContainer(AbstractFileStoreWrite.java:447)
	at org.apache.paimon.operation.AbstractFileStoreWrite.lambda$getWriterWrapper$2(AbstractFileStoreWrite.java:409)
	at java.util.HashMap.computeIfAbsent(HashMap.java:1127)
	at org.apache.paimon.operation.AbstractFileStoreWrite.getWriterWrapper(AbstractFileStoreWrite.java:408)
	at org.apache.paimon.operation.AbstractFileStoreWrite.write(AbstractFileStoreWrite.java:142)
	at org.apache.paimon.table.sink.TableWriteImpl.writeAndReturn(TableWriteImpl.java:175)
	at org.apache.paimon.flink.sink.StoreSinkWriteImpl.write(StoreSinkWriteImpl.java:179)
	at org.apache.paimon.flink.sink.RowDataStoreWriteOperator.processElement(RowDataStoreWriteOperator.java:128)
	... 13 more
Caused by: org.apache.paimon.shade.guava30.com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: JavaCodeSplitter failed. This is a bug. Please file an issue.
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache.get(LocalCache.java:3962)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4859)
	at org.apache.paimon.codegen.CodeGenUtils.generate(CodeGenUtils.java:109)
	... 28 more
Caused by: java.lang.RuntimeException: JavaCodeSplitter failed. This is a bug. Please file an issue.
	at org.apache.paimon.codegen.codesplit.JavaCodeSplitter.split(JavaCodeSplitter.java:35)
	at org.apache.paimon.codegen.GeneratedClass.<init>(GeneratedClass.java:57)
	at org.apache.paimon.codegen.EqualiserCodeGenerator.generateRecordEqualiser(EqualiserCodeGenerator.scala:79)
	at org.apache.paimon.codegen.CodeGeneratorImpl.generateRecordEqualiser(CodeGeneratorImpl.java:58)
	at org.apache.paimon.codegen.CodeGenUtils.lambda$newRecordEqualiser$4(CodeGenUtils.java:97)
	at org.apache.paimon.codegen.CodeGenUtils.lambda$generate$5(CodeGenUtils.java:112)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$LocalManualCache$1.load(LocalCache.java:4864)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
	at org.apache.paimon.shade.guava30.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
	... 31 more
Caused by: java.lang.Error: Unresolved compilation problems: 
	JavaLexer cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParserBaseVisitor cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParserBaseVisitor cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParserBaseVisitor cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParser cannot be resolved to a type
	JavaParser cannot be resolved to a type

	at org.apache.paimon.codegen.codesplit.ReturnValueRewriter.<init>(ReturnValueRewriter.java:75)
	at org.apache.paimon.codegen.codesplit.JavaCodeSplitter.splitImpl(JavaCodeSplitter.java:49)
	at org.apache.paimon.codegen.codesplit.JavaCodeSplitter.split(JavaCodeSplitter.java:33)
	... 41 more

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions