master/1.x branch does not compile with java9

[rmuir]

master works fine. Maybe there is a real bug here, e.g. wildcard imports, and a new class in java9 causes a conflict? This is why i hate wildcard imports :)

Otherwise it could be a compiler bug or something:

main:
[echo] Using OpenJDK Runtime Environment 1.9.0-internal-rmuir_2015_03_18_00_27-b00 Oracle Corporation
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (invalid-patterns) @ elasticsearch ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO]
[INFO] --- buildnumber-maven-plugin:1.2:create (default) @ elasticsearch ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] Executing: /bin/sh -c cd /home/rmuir/workspace/elasticsearch && git rev-parse --verify HEAD
[INFO] Working directory: /home/rmuir/workspace/elasticsearch
[INFO] Storing buildNumber: 78b5a6a at timestamp: 1426705014243
[INFO] Executing: /bin/sh -c cd /home/rmuir/workspace/elasticsearch && git rev-parse --verify HEAD
[INFO] Working directory: /home/rmuir/workspace/elasticsearch
[INFO] Storing buildScmBranch: UNKNOWN
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ elasticsearch ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] Copying 5 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ elasticsearch ---
[INFO] Compiling 3171 source files to /home/rmuir/workspace/elasticsearch/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/inject/Scopes.java:[112,23] error: method scope in interface Scope cannot be applied to given types;
[ERROR] required: Key<T#1>,Provider<T#1>
found: Key<T#2>,ProviderToInternalFactoryAdapter<CAP#1>
reason: inferred type does not conform to equality constraint(s)
inferred: CAP#1
equality constraints(s): CAP#1,T#2
where T#1,T#2 are type-variables:
T#1 extends Object declared in method <T#1>scope(Key<T#1>,Provider<T#1>)
T#2 extends Object declared in method <T#2>scope(Key<T#2>,InjectorImpl,InternalFactory<? extends T#2>,Scoping)
where CAP#1 is a fresh type-variable:
CAP#1 extends T#2 from capture of ? extends T#2
/home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/collect/Iterators2.java:[34,63] error: incompatible types: PeekingIterator<CAP#1> cannot be converted to PeekingIterator
[ERROR] where T is a type-variable:
T extends Object declared in method deduplicateSorted(Iterator<? extends T>,Comparator<? super T>)
where CAP#1 is a fresh type-variable:
CAP#1 extends T from capture of ? extends T
/home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/inject/util/Modules.java:[92,58] error: incompatible types: ImmutableSet<CAP#1> cannot be converted to Set
[ERROR] where CAP#1 is a fresh type-variable:
CAP#1 extends Module from capture of ? extends Module
/home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/inject/util/Modules.java:[123,50] error: incompatible types: ImmutableSet<CAP#1> cannot be converted to ImmutableSet
[INFO] 4 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26.162 s
[INFO] Finished at: 2015-03-18T14:57:07-05:00
[INFO] Final Memory: 23M/531M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project elasticsearch: Compilation failure: Compilation failure:
[ERROR] /home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/inject/Scopes.java:[112,23] error: method scope in interface Scope cannot be applied to given types;
[ERROR] required: Key<T#1>,Provider<T#1>
[ERROR] found: Key<T#2>,ProviderToInternalFactoryAdapter<CAP#1>
[ERROR] reason: inferred type does not conform to equality constraint(s)
[ERROR] inferred: CAP#1
[ERROR] equality constraints(s): CAP#1,T#2
[ERROR] where T#1,T#2 are type-variables:
[ERROR] T#1 extends Object declared in method <T#1>scope(Key<T#1>,Provider<T#1>)
[ERROR] T#2 extends Object declared in method <T#2>scope(Key<T#2>,InjectorImpl,InternalFactory<? extends T#2>,Scoping)
[ERROR] where CAP#1 is a fresh type-variable:
[ERROR] CAP#1 extends T#2 from capture of ? extends T#2
[ERROR] /home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/collect/Iterators2.java:[34,63] error: incompatible types: PeekingIterator<CAP#1> cannot be converted to PeekingIterator
[ERROR] where T is a type-variable:
[ERROR] T extends Object declared in method deduplicateSorted(Iterator<? extends T>,Comparator<? super T>)
[ERROR] where CAP#1 is a fresh type-variable:
[ERROR] CAP#1 extends T from capture of ? extends T
[ERROR] /home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/inject/util/Modules.java:[92,58] error: incompatible types: ImmutableSet<CAP#1> cannot be converted to Set
[ERROR] where CAP#1 is a fresh type-variable:
[ERROR] CAP#1 extends Module from capture of ? extends Module
[ERROR] /home/rmuir/workspace/elasticsearch/src/main/java/org/elasticsearch/common/inject/util/Modules.java:[123,50] error: incompatible types: ImmutableSet<CAP#1> cannot be converted to ImmutableSet
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

[rmuir]

We hit a similar issue in lucene when upgrading. @uschindler is filing a bug at oracle.

[uschindler]

I submitted a bug report! Its is not yet visible, I will report back. The problem is caused by the diamond operator with -source/target 1.7. The diamond's real type is deducted not from left side of assignement as JLS spec says, but instead from the parameter passed to the constructor (e.g. new HashSet<>(otherSet)). In that case it takes type from otherSet which is wrong.

[rmuir]

Thanks for digging Uwe. I tried to understand the problems, I felt like i was looking at c++ compiler errors with tons of templates :)

[uschindler]

Link to Lucene: http://mail-archives.apache.org/mod_mbox/lucene-dev/201503.mbox/%3C066501d06256%242f949300%248ebdb900%24%40thetaphi.de%3E

[uschindler]

Hi,
the bug report at Oracle is: https://bugs.openjdk.java.net/browse/JDK-8075793

[s1monw]

thanks @uschindler

[uschindler]

Hi,
according to the OpenJDK people, this is not really a bug in the JDK, because the Java Language Spec was implemented in a wrong way up to Java 8. The current code is only valid according to JLS with Java 8+ (where the JLS was adopted to allow this behaviour that we expected) - this is why it works in Lucene trunk with Java 8. It was just a bug in the Java 5, 6, 7 compilers that did a "wrong"shortcut to get the type of the diamond.

This wrong behaviour in Javac is fixed in Java 9, so the broken code should only compile with -source 8 or -source 9 (LOL).

In fact you have to add the type into the diamond for Java 7, so basically the code on our side was wrong (I did not verify that from the JLS, this is what was written by Dan Smith).

Dan Smith wrote:

Simplified test:

  class C<T> {}

  <T> C<T> m(C<? extends T> x) { return null; }

  void test(C<? extends Number> arg) {
    C<Number> c = m(arg);
  }

Type checking behavior is clear, per JLS 3 and JLS 7:

• The type of 'arg' is C<CAP#1>, where CAP#1 extends Number (6.5.6.1)
• Inference determines that T has lower bound CAP#1 (15.12.2.7)
• The type of T is inferred as CAP#1 (15.12.2.7)
• The type of the method invocation ('m(arg)') is C<CAP#1> (15.12.2.6)
• C<CAP#1> cannot be assigned to C (4.10.2)

Traditionally, javac has performed some unspecified "simplifications" that led to T being inferred as 'Number' rather than 'CAP#1'. These unspecified behaviors break programs that should compile, but also allow programs like these to be compiled when the spec disallows them (see JDK-8039214).

For the class of programs described by this bug, changes made to inference in JLS 8 make these JLS/javac inconsistencies moot, because both strategies get the same answer. But under -source 7, the distinction is apparent, and fixing JDK-8039214 exposes these programs' dependency on incorrect type checking behavior.

Broadly, our choices:

• Not A Bug, expect users to fix their code
• Freeze the -source 7 type checking implementation, and stop fixing bugs in it (hard to separate JDK-8039214 from many other bugs...)
• Retroactively "specify" javac's legacy behavior as part of inference in 5-7, re-implement it, and hope this doesn't cause subtle breakage elsewhere

[uschindler]

Because it is unclear, how this issue is solved on the OpenJDK side, I would recommend to stay with the current fix, because those diamonds and type guessing by the compiler was always broken (my personal opinion). There also exist differences between Eclipse's compiler and JDK's and most of those differences are around this type of stuff.

I always hated diamonds and was always more happy with being explicit. So we should be more explicit in our code, if the type inference is not easy to explain!

Thanks,
the Generics Policeman :-)