Skip to content

[AMORO-3853] Support to build amoro with Spark-3.5 + Java17#4029

Closed
turboFei wants to merge 6 commits intoapache:masterfrom
turboFei:spark_35_jdk17
Closed

[AMORO-3853] Support to build amoro with Spark-3.5 + Java17#4029
turboFei wants to merge 6 commits intoapache:masterfrom
turboFei:spark_35_jdk17

Conversation

@turboFei
Copy link
Member

@turboFei turboFei commented Dec 29, 2025

Why are the changes needed?

Close #3853

Brief change log

  • Support to build amoro with Spark-3.5 + Java17

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (yes / no)
  • If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@github-actions github-actions bot added module:mixed-flink Flink moduel for Mixed Format module:ams-server Ams server module type:infra type:build labels Dec 29, 2025
@turboFei turboFei changed the title Spark 35 jdk17 [AMORO-3853] Support to build amoro with Spark-3.5 + Java17 Dec 29, 2025
equals = DateTimeUtil.microsFromTimestamptz((OffsetDateTime) o1) == (Long) o2;
} else {
equals = false;
}
Copy link
Member Author

@turboFei turboFei Dec 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To fix Error:

2025-10-31T21:27:36.7841941Z [ERROR] TestKeyedContinuousOptimizing.run  Time elapsed: 0.65 s  <<< ERROR!
2025-10-31T21:27:36.7842810Z java.lang.IllegalStateException: Not an instance of java.lang.Long: 2019-01-26T21:27:34.557898
2025-10-31T21:27:36.7843640Z 	at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:123)
2025-10-31T21:27:36.7844403Z 	at org.apache.amoro.io.writer.RecordWithAction.get(RecordWithAction.java:68)
2025-10-31T21:27:36.7845027Z 	at org.apache.iceberg.types.Comparators$StructLikeComparator.compare(Comparators.java:111)
2025-10-31T21:27:36.7845597Z 	at org.apache.iceberg.types.Comparators$StructLikeComparator.compare(Comparators.java:84)
2025-10-31T21:27:36.7846143Z 	at org.apache.iceberg.util.StructLikeWrapper.equals(StructLikeWrapper.java:90)
2025-10-31T21:27:36.7846570Z 	at java.base/java.util.HashMap.getNode(HashMap.java:570)
2025-10-31T21:27:36.7846936Z 	at java.base/java.util.HashMap.containsKey(HashMap.java:594)
2025-10-31T21:27:36.7847297Z 	at java.base/java.util.HashSet.contains(HashSet.java:205)
2025-10-31T21:27:36.7847701Z 	at org.apache.iceberg.util.StructLikeSet.contains(StructLikeSet.java:61)
2025-10-31T21:27:36.7848281Z 	at org.apache.amoro.server.optimizing.flow.view.KeyedTableDataView.match(KeyedTableDataView.java:234)
2025-10-31T21:27:36.7849179Z 	at org.apache.amoro.server.optimizing.flow.checker.DataConcurrencyChecker.check(DataConcurrencyChecker.java:70)
2025-10-31T21:27:36.7849932Z 	at org.apache.amoro.server.optimizing.flow.CompleteOptimizingFlow.check(CompleteOptimizingFlow.java:159)
2025-10-31T21:27:36.7850657Z 	at org.apache.amoro.server.optimizing.flow.CompleteOptimizingFlow.optimize(CompleteOptimizingFlow.java:126)
2025-10-31T21:27:36.7851685Z 	at org.apache.amoro.server.optimizing.flow.TestKeyedContinuousOptimizing.mustFullCycle(TestKeyedContinuousOptimizing.java:172)
2025-10-31T21:27:36.7852569Z 	at org.apache.amoro.server.optimizing.flow.TestKeyedContinuousOptimizing.run(TestKeyedContinuousOptimizing.java:136)
2025-10-31T21:27:36.7853244Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2025-10-31T21:27:36.7853848Z 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2025-10-31T21:27:36.7854547Z 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2025-10-31T21:27:36.7855198Z 	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
2025-10-31T21:27:36.7855672Z 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
2025-10-31T21:27:36.7856231Z 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
2025-10-31T21:27:36.7856794Z 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
2025-10-31T21:27:36.7857346Z 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
2025-10-31T21:27:36.7857889Z 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
2025-10-31T21:27:36.7858411Z 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
2025-10-31T21:27:36.7858934Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
2025-10-31T21:27:36.7859379Z 	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
2025-10-31T21:27:36.7859862Z 	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
2025-10-31T21:27:36.7860347Z 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
2025-10-31T21:27:36.7860833Z 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
2025-10-31T21:27:36.7861561Z 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
2025-10-31T21:27:36.7862041Z 	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
2025-10-31T21:27:36.7862434Z 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
2025-10-31T21:27:36.7862858Z 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
2025-10-31T21:27:36.7863361Z 	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
2025-10-31T21:27:36.7863772Z 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
2025-10-31T21:27:36.7864173Z 	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
2025-10-31T21:27:36.7864526Z 	at org.junit.runners.Suite.runChild(Suite.java:128)
2025-10-31T21:27:36.7864846Z 	at org.junit.runners.Suite.runChild(Suite.java:27)
2025-10-31T21:27:36.7865191Z 	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
2025-10-31T21:27:36.7865581Z 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
2025-10-31T21:27:36.7865991Z 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
2025-10-31T21:27:36.7866411Z 	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
2025-10-31T21:27:36.7866815Z 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
2025-10-31T21:27:36.7867238Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
2025-10-31T21:27:36.7867686Z 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
2025-10-31T21:27:36.7868073Z 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
2025-10-31T21:27:36.7868576Z 	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
2025-10-31T21:27:36.7868974Z 	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
2025-10-31T21:27:36.7869335Z 	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
2025-10-31T21:27:36.7869662Z 	at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
2025-10-31T21:27:36.7870094Z 	at org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
2025-10-31T21:27:36.7870681Z 	at org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
2025-10-31T21:27:36.7871248Z 	at org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
2025-10-31T21:27:36.7872048Z 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:147)
2025-10-31T21:27:36.7872807Z 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:127)
2025-10-31T21:27:36.7873557Z 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:90)
2025-10-31T21:27:36.7874420Z 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:55)
2025-10-31T21:27:36.7875270Z 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:102)
2025-10-31T21:27:36.7876088Z 	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:54)
2025-10-31T21:27:36.7876740Z 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
2025-10-31T21:27:36.7877281Z 	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
2025-10-31T21:27:36.7877957Z 	at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
2025-10-31T21:27:36.7878630Z 	at org.apache.maven.surefire.junitplatform.LazyLauncher.execute(LazyLauncher.java:55)
2025-10-31T21:27:36.7879277Z 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.execute(JUnitPlatformProvider.java:223)
2025-10-31T21:27:36.7880043Z 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:175)
2025-10-31T21:27:36.7880792Z 	at org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:139)
2025-10-31T21:27:36.7881745Z 	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:456)
2025-10-31T21:27:36.7882712Z 	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:169)
2025-10-31T21:27:36.7883556Z 	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
2025-10-31T21:27:36.7884482Z 	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)

Related iceberg issue:

apache/iceberg#6510

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@turboFei
Copy link
Member Author

For OpenAPI SDK GA failure, should be fixed by #4028

@turboFei turboFei marked this pull request as draft December 30, 2025 05:17
@turboFei turboFei closed this Dec 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:ams-server Ams server module module:mixed-flink Flink moduel for Mixed Format type:build type:infra

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement]: Support to build amoro on java 17

1 participant