[Task]: Upgrade checkerframework to 3.39.0+

[Abacn]

What needs to happen?

Currently we use checkerframework 3.27.0 which supports up to Java 17. When run with Java 21, it will complain and due to -Werror it fails compile.

The latest checkerframework 3.39.0 adds (preliminary) support of Java21. However, it is found that checkerframework crashes when compiling beam (see stacktrace below), in addition to a number of new check violations.

> Task :sdks:java:core:compileJava
error: StructuralEqualityComparer: unexpected combination:
    type: TYPEVAR OutputT extends Object
    superType: WILDCARD ? extends Object
    visitHistory = org.checkerframework.framework.type.StructuralEqualityVisitHistory@3599a2a8
  ; The Checker Framework crashed.  Please report the crash.  Version: Checker Framework 3.39.0. 
  Compilation unit: /Users/yathu/beam/sdks/java/core/src/main/java/org/apache/beam/sdk/io/FileIO.java
  Last visited tree at line 1090 column 14:
        return toBuilder().setSinkFn((Contextful) sinkFn).setOutputFn(outputFn).build();
  Exception: java.lang.Throwable; java.lang.Throwable
        at org.checkerframework.javacutil.BugInCF.<init>(BugInCF.java:22)
        at org.checkerframework.framework.type.visitor.AtmComboVisitor.defaultAction(AtmComboVisitor.java:58)
        at org.checkerframework.framework.type.StructuralEqualityComparer.defaultAction(StructuralEqualityComparer.java:56)
        at org.checkerframework.framework.type.StructuralEqualityComparer.defaultAction(StructuralEqualityComparer.java:29)
        at org.checkerframework.framework.type.visitor.AbstractAtmComboVisitor.visitTypevar_Wildcard(AbstractAtmComboVisitor.java:560)
        at org.checkerframework.framework.util.AtmCombo.accept(AtmCombo.java:620)
        at org.checkerframework.framework.type.StructuralEqualityComparer.areEqual(StructuralEqualityComparer.java:91)
        at org.checkerframework.framework.type.StructuralEqualityComparer.visitDeclared_Declared(StructuralEqualityComparer.java:246)
        at org.checkerframework.framework.type.StructuralEqualityComparer.visitDeclared_Declared(StructuralEqualityComparer.java:29)
        at org.checkerframework.framework.util.AtmCombo.accept(AtmCombo.java:312)
        at org.checkerframework.framework.type.StructuralEqualityComparer.areEqual(StructuralEqualityComparer.java:91)
        at org.checkerframework.framework.type.StructuralEqualityComparer.areEqualInHierarchy(StructuralEqualityComparer.java:101)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.areEqualInHierarchy(DefaultTypeHierarchy.java:338)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.isContainedBy(DefaultTypeHierarchy.java:413)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.isContainedMany(DefaultTypeHierarchy.java:647)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.visitTypeArgs(DefaultTypeHierarchy.java:624)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.visitDeclared_Declared(DefaultTypeHierarchy.java:550)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.visitDeclared_Declared(DefaultTypeHierarchy.java:48)
        at org.checkerframework.framework.util.AtmCombo.accept(AtmCombo.java:312)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.isSubtype(DefaultTypeHierarchy.java:258)
        at org.checkerframework.framework.type.DefaultTypeHierarchy.isSubtype(DefaultTypeHierarchy.java:136)
        at org.checkerframework.common.basetype.BaseTypeVisitor.commonAssignmentCheck(BaseTypeVisitor.java:3043)
        at org.checkerframework.common.basetype.BaseTypeVisitor.commonAssignmentCheck(BaseTypeVisitor.java:3018)
        at org.checkerframework.common.basetype.BaseTypeVisitor.checkArguments(BaseTypeVisitor.java:3585)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitMethodInvocation(BaseTypeVisitor.java:1792)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitMethodInvocation(BaseTypeVisitor.java:186)
        at com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1644)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:186)
        at com.sun.source.util.TreeScanner.visitMemberSelect(TreeScanner.java:680)
        at com.sun.tools.javac.tree.JCTree$JCFieldAccess.accept(JCTree.java:2112)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitMethodInvocation(BaseTypeVisitor.java:1829)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitMethodInvocation(BaseTypeVisitor.java:186)
        at com.sun.tools.javac.tree.JCTree$JCMethodInvocation.accept(JCTree.java:1644)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:186)
        at com.sun.source.util.TreeScanner.visitReturn(TreeScanner.java:469)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitReturn(BaseTypeVisitor.java:2200)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitReturn(BaseTypeVisitor.java:186)
        at com.sun.tools.javac.tree.JCTree$JCReturn.accept(JCTree.java:1548)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:186)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
        at com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
        at com.sun.source.util.TreeScanner.visitBlock(TreeScanner.java:248)
        at com.sun.tools.javac.tree.JCTree$JCBlock.accept(JCTree.java:1026)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:186)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
        at com.sun.source.util.TreeScanner.visitMethod(TreeScanner.java:206)
        at org.checkerframework.framework.source.SourceVisitor.visitMethod(SourceVisitor.java:110)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitMethod(BaseTypeVisitor.java:1043)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitMethod(BaseTypeVisitor.java:186)
        at com.sun.tools.javac.tree.JCTree$JCMethodDecl.accept(JCTree.java:898)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:186)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
        at com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
        at com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
        at org.checkerframework.framework.source.SourceVisitor.visitClass(SourceVisitor.java:98)
        at org.checkerframework.common.basetype.BaseTypeVisitor.processClassTree(BaseTypeVisitor.java:612)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:558)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:186)
        at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:808)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:82)
        at org.checkerframework.framework.source.SourceVisitor.scan(SourceVisitor.java:92)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:405)
        at org.checkerframework.common.basetype.BaseTypeVisitor.scan(BaseTypeVisitor.java:186)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:90)
        at com.sun.source.util.TreeScanner.scan(TreeScanner.java:105)
        at com.sun.source.util.TreeScanner.scanAndReduce(TreeScanner.java:113)
        at com.sun.source.util.TreeScanner.visitClass(TreeScanner.java:187)
        at org.checkerframework.framework.source.SourceVisitor.visitClass(SourceVisitor.java:98)
        at org.checkerframework.common.basetype.BaseTypeVisitor.processClassTree(BaseTypeVisitor.java:612)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:558)
        at org.checkerframework.common.basetype.BaseTypeVisitor.visitClass(BaseTypeVisitor.java:186)
        at com.sun.tools.javac.tree.JCTree$JCClassDecl.accept(JCTree.java:808)
        at com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:56)
        at org.checkerframework.framework.source.SourceVisitor.visit(SourceVisitor.java:86)
        at org.checkerframework.framework.source.SourceChecker.typeProcess(SourceChecker.java:1046)
        at org.checkerframework.common.basetype.BaseTypeChecker.typeProcess(BaseTypeChecker.java:558)
        at org.checkerframework.common.basetype.BaseTypeChecker.typeProcess(BaseTypeChecker.java:551)
        at org.checkerframework.javacutil.AbstractTypeProcessor$AttributionTaskListener.finished(AbstractTypeProcessor.java:188)
        at com.sun.tools.javac.api.ClientCodeWrapper$WrappedTaskListener.finished(ClientCodeWrapper.java:828)
        at com.sun.tools.javac.api.MultiTaskListener.finished(MultiTaskListener.java:120)
        at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1404)
        at com.sun.tools.javac.main.JavaCompiler.flow(JavaCompiler.java:1353)
        at com.sun.tools.javac.main.JavaCompiler.compile(JavaCompiler.java:946)
        at com.sun.tools.javac.api.JavacTaskImpl.lambda$doCall$0(JavacTaskImpl.java:100)
        at com.sun.tools.javac.api.JavacTaskImpl.handleExceptions(JavacTaskImpl.java:142)
        at com.sun.tools.javac.api.JavacTaskImpl.doCall(JavacTaskImpl.java:96)
        at com.sun.tools.javac.api.JavacTaskImpl.call(JavacTaskImpl.java:90)
        at org.gradle.internal.compiler.java.IncrementalCompileTask.call(IncrementalCompileTask.java:92)

Issue Priority

Priority: 2 (default / most normal work should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner