Initial Spike to support JDK 9 #1994

kittylyst · 2017-05-13T20:04:18Z

This patch is a first step to supporting Java 9. It fixes some compile errors that the tighter Java 9 compiler insists upon, and refactors classes into the module structure. More work is still needed though - so it may be better on a branch rather than mainline.

Reimplemented Collections.scan methods

Moved not-null check to some helper methods, ordered methods by name

* Map.ofAll(Stream, Mapper, Mapper) #1341 * Map.ofAll(Stream, Mapper) #1341 * formatting * rearrange methods

Updated pom version to 2.1.0

…hods) on my local machine

Moved javaslang gwt

Expand TupleTest for Tuple7 and Tuple8

Fix #1318

Add autogenerated Tuple narrow methods for issue #1663

… hint

General `Vector` speedup

Added Stream.concat() methods

introduces Ordered interface #1495

Implement Seq.shuffle()

nfekete · 2017-05-13T20:08:49Z

If we compile this code, will it work with Java8? (i.e. backward compatible on bytecode level)

kittylyst · 2017-05-13T20:27:04Z

I do not believe so. What todays experiments have shown me leads me to think that there is no way that a warnings == fatal project can have a source base that compiles cleanly under both JDK 8 & 9. It seems that the javac compiler is much more strongly-conforming in 9 than 8.

nfekete · 2017-05-13T21:48:13Z

Shouldn't then we keep our primary JDK version at 8, so that we can use it both in JRE8 and 9? I guess at least the bytecode compiled with JDK8 will be forward compatible with JDK9. Or am I wrong?

kittylyst · 2017-05-14T09:34:38Z

OK, here's a breakdown of what's in the PR, so you know which bits you might want:

.gitignore - just ignore vi swap files : SAFE for both 8 & 9
file renames & module-info.java : only needed if having a fork or branch that is JDK 9
pom.xml - only needed if having a fork or branch that is JDK 9. Probably a good idea to move to Scala 2.12 soon though, even for Java 8
Changes to the GwtIncompatible.java files - SAFE for both 8 & 9 : not needed for 8 but the current situation is actually incorrect and the 9 compiler exposes it.
io/vavr/Lazy.java - NOT SAFE to backport to 8. 8 is actually incorrect and the 9 compiler exposes it.
io/vavr/AssertionsExtensions.java : SAFE for both 8 & 9. 9 changes behaviour so that this method will emit a deprecation warning whereas 8 won't
io/vavr/MatchTest.java - I have no idea what the correct form for this for 9 is. The current 8 code is wrong, and relies on a type inferencing bug in javac. I could not find a correct form for 9, and exposed a javac bug in 9 when trying to forward port it. The current code is unsafe and almost certainly doesn't mean what you think it does.

I know this isn't a great story for you, but please consider how you might support 9 going forwards. Vavr is a great project, and as it has so few dependencies, it could be made into a set of explicit modules, which would be great to see. Thanks for all the work so far!

danieldietrich · 2017-05-16T07:37:48Z

Hi Ben,

thank you for the PR. It is great to see a JCP member here!

So far, I haven't found a reason to bump Vavr's binary code to the upcoming Java version. JDK9 doesn't offer any features that will enhance our API. If we would use modules under the hood, we would just ensure that we expose the right things to the outside. Currently we use workarounds for that purpose, like auxiliary classes and package private classes (I know, even if they are not part of the contract, they are currently discoverable via reflection).

However, it is important that Java 9 applications are able to use the Vavr binaries compiled with Java 8. This is one thing that has to be investigated. I haven't found the time to get my hands on Java 9, yet (besides jshell - I love it).

Thank you also for your summary. One thing I don't like to see is that GWT will not be able any more to find the package-private GwtIncompatible annotation, which I do not want to expose to the outside. It is also no option to pull in a 3rd party library to get this annotation, Vavr is a zero-dependency library.

It really surprises me that Lazy and MatchTest (resp. our pattern matching API) are incorrect. We put much effort in the implementation and from my point of view all makes perfectly sense. For example I tested pretty well that types are recursively inferred along the pattern cascade. Please note that the match cases are partial functions. I.e. a pattern might not match if the given sub-types do not match the actual types. Especially we did not implement the visitor pattern, we did something more general. It is like Scala but without exhaustiveness checks by the compiler.

The current code is unsafe and almost certainly doesn't mean what you think it does.

Is it the exhaustiveness you address? Could you please elaborate on this?

I could not find a correct form for 9, and exposed a javac bug in 9 when trying to forward port it.

Did you already filed a bug on Oracles bug-tracker?

I think I have to dive deeper into Java 9 in order to understand what went wrong with Lazy and Match. Your investigation makes perfectly sense. We will see how we can benefit from Java modules. At the latest we will jump to the next binary version with Java 10, there are a lot of interesting features we are waiting for.

Fixed Jigsaw args sent to maven compiler in the child POMs

Update pom.xml

Move sources to correct level

kittylyst · 2017-05-22T13:42:29Z

Hi Daniel - thanks for the comments on the PR. We are making progress on this - including having collaborated to help get a version of maven-compiler-plugin (I think it will be 3.6.2) that works with 9. Currently, having problems with maven-bundle-plugin. The problem seems to be in bndtools - and some folk from the OSGi community are helping us look at the issue.

I think the current issue we're facing is related to this thread - #1225 - is there a workaround that doesn't involve using the maven-bundle-plugin ? - as apparently it will be a while before it gets updated to handle JDK 9.

I started a thread about this here: https://groups.google.com/forum/#!forum/bndtools-users - maybe you could come & join?

Will respond to the API and other points in a separate comment - getting the build working is my top priority because of the time-sensitivity.

The 3.4.0-SNAPSHOT build will not fail on module-info.class files.

Use latest Bnd snapshot

kittylyst · 2017-05-25T09:01:02Z

Just to follow up on this:

There are now fixes in place for maven-compiler-plugin and BndTools that have progressed the build. However, the test cases are still failing to compile. The issue appears to be within Vavr itself, to do with the generated test cases.

As I'm not too familiar with the source generation that Vavr does yet - if someone has some cycles to help me look at this, it would be greatly appreciated.

danieldietrich · 2017-06-13T14:28:33Z

@kittylyst thank you for all your effort! Currently I'm on vacation (until end of June). Then I will help you on this issue. Currently I have to catch up with all the pending comments and pull requests...

I have some questions:

You said "getting the build working is my top priority because of the time-sensitivity". Why is it time-sensitive? (JDK 9 GA is deferred to 21 Sep '17)
I do not expect (enterprise) users to early adopt Java 9 when it is released. Too many open tooling questions, library incompatibilities etc. My goal is to keep Vavr Java 8 compatible. Do I understand correctly, that this PR isn't Java 8 compatible any more?

However, I think I need to get my hands dirty in order to understand the impact :)

paplorinc · 2017-07-14T18:23:06Z

pom.xml

        <jmh.version>1.18</jmh.version>
        <junit.version>4.12</junit.version>
        <gwt.version>2.8.0</gwt.version>
        <maven.build-helper.version>1.12</maven.build-helper.version>
-        <maven.bundle.version>3.2.0</maven.bundle.version>
+        <maven.bundle.version>3.3.0</maven.bundle.version>


If you separate these in another PR - so that it stays Java 8 & 9 compatible - I'm sure that will be merged :)

danieldietrich · 2017-07-15T08:27:00Z

This PR is important! Currently it is not priority 1 but it will flow into the master at some point...

paplorinc

Good job, though I would definitely separate it into multiple pull requests, as it obviously cannot be merged in this way (e.g. it's not Java 8 compatible).

paplorinc · 2017-09-20T16:25:54Z

pom.xml

@@ -87,8 +87,26 @@ Note: The maven build currently needs to be started from the vavr root dir
        <maven.exec.version>1.5.0</maven.exec.version>
        <maven.gwt.plugin>1.0-rc-6</maven.gwt.plugin>
        <scala.maven.version>3.2.2</scala.maven.version>
-        <scala.version>2.11.7</scala.version>
-    </properties>
+        <scala.version>2.12.2</scala.version>


👍 for Scala update
(code generation crashed otherwise anyway)

paplorinc · 2017-09-20T16:25:54Z

vavr/src/main/java/io/vavr/Lazy.java

@@ -101,7 +101,7 @@ private Lazy(Supplier<? extends T> supplier) {
     */
    public static <T> Lazy<Seq<T>> sequence(Iterable<? extends Lazy<? extends T>> values) {
        Objects.requireNonNull(values, "values is null");
-        return Lazy.of(() -> Vector.ofAll(values).map(Lazy::get));
+        return Lazy.of(() -> Vector.ofAll(values).map(Lazy<? extends T>::get));


i think this is a bug in both the JDK and in Idea, they report different things for every variation of the code

An alternative suggestion that Idea accepts also:

@SuppressWarnings("Convert2MethodRef") // TODO fixme on Java 9 public static <T> Lazy<Seq<T>> sequence(Iterable<? extends Lazy<? extends T>> values) { Objects.requireNonNull(values, "values is null"); return Lazy.of(() -> Vector.ofAll(values).map(v -> v.get())); }

paplorinc · 2017-09-20T16:25:54Z

vavr/src/test/java/io/vavr/AssertionsExtensions.java

@@ -27,6 +27,7 @@ public static ClassAssert assertThat(Class<?> clazz) {
            this.clazz = clazz;
        }

+        @SuppressWarnings("deprecation")


~~instead of this I would recommend using the new canAccess method, something like:~~

public void isNotInstantiable() { try { Assertions.assertThat(clazz.getDeclaredConstructor().canAccess(null)).isFalse(); } catch (NoSuchMethodException e) { throw new AssertionError("no default constructor found"); } }

Edit: This isn't java 8 compatible, my mistake

paplorinc · 2017-09-20T16:25:54Z

vavr/src/test/java/io/vavr/MatchTest.java

-        assertThat(actual).isEqualTo("Some(1)::List(Some(2.0))");
-    }
-
+//    @SuppressWarnings("UnnecessaryLocalVariable")


@danieldietrich, tomorrow is the release date of JDK 9, this commit solves most of the compile problems (I basically recreated everything to validate it, good job, guys (there are two commiters, apparently).
If you could fix these (it's weird generics magic that I didn't have the courage to understand), we could get on the Java 9 wave(r).

paplorinc · 2017-09-20T16:25:54Z

.gitignore

@@ -29,3 +29,6 @@ hs_err_pid*

 # -- Emacs
 .#*
+


seems unrelated

paplorinc · 2017-09-20T16:25:54Z

pom.xml

@@ -66,15 +66,15 @@ Note: The maven build currently needs to be started from the vavr root dir
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <assertj.core.version>3.6.2</assertj.core.version>
        <eclipse.lifecycle.mapping.version>1.0.0</eclipse.lifecycle.mapping.version>
-        <java.version>1.8</java.version>
+        <java.version>9</java.version>


It won't be merged this way, since it has to be Java 8 compatible also.

paplorinc · 2017-09-20T16:25:54Z

vavr/src/main/java/module-info.java

@@ -0,0 +1,7 @@
+module io.vavr {


Is this still Java 8 compatible?
@danieldietrich do we need a separate, modularized Java 9 release also?

paplorinc · 2017-09-23T12:53:42Z

pom.xml

        <maven.clean.version>3.0.0</maven.clean.version>
        <maven.install.version>2.5.2</maven.install.version>
-        <maven.compiler.version>3.6.0</maven.compiler.version>
+        <maven.compiler.version>3.6.2-SNAPSHOT</maven.compiler.version>


3.7.0 is already available and it solves the compiler exception also

paplorinc · 2017-09-23T12:53:42Z

vavr/src/test/java/io/vavr/MatchTest.java

+//    public void shouldDecomposeListWithNonEmptyTail() {
+//        final List<Option<Number>> intOptionList = List.of(Option.some(1), Option.some(2.0));
+//        final String actual = Match(intOptionList).of(
+//                Case($Cons($Some($(1)), $Cons($Some($(2.0)), $())), (x, xs) -> {


The following seems to work:

Case($Cons($Some($(1)), $Cons($Some($(2.0)), $())), (Some<Number> x, List<Option<Number>> xs) -> x + "::" + xs)

paplorinc · 2017-09-23T12:53:42Z

pom.xml

@@ -279,6 +297,13 @@ Note: The maven build currently needs to be started from the vavr root dir
                            </goals>
                        </execution>
                    </executions>
+                    <dependencies>


why was this needed?

danieldietrich and others added 30 commits November 23, 2016 09:27

Merge pull request #1684 from ashrko619/scan-methods

765071b

Reimplemented Collections.scan methods

Moved not-null check to some helper methods, ordered methods by name

ce39e0b

Merge pull request #1703 from danieldietrich/helper-maintenance

68c71aa

Moved not-null check to some helper methods, ordered methods by name

Map.ofAll(Stream, Mapper) #1341 (#1706)

0cf03b4

* Map.ofAll(Stream, Mapper, Mapper) #1341 * Map.ofAll(Stream, Mapper) #1341 * formatting * rearrange methods

Updated pom version to 2.1.0

682d0ea

Merge pull request #1707 from danieldietrich/pom-version-update

f4ed1fc

Updated pom version to 2.1.0

Removed javaslang-pure

7948682

Reactivated benchmark profile, updated assertj & jmh plugin versions

cedb746

Moved GWT modules to their own Repo

c84f8bc

Added Javaslang file header

d992a5e

speeds up build by 27% by using 4 threads for unit tests (classes/met…

2d3482b

…hods) on my local machine

Merge pull request #1708 from danieldietrich/moved-javaslang-gwt

b72230a

Moved javaslang gwt

introduces Ordered interface #1495

097612c

Add autogenerated Tuple narrow methods for issue #1663

12775b8

Expand TupleTest for Tuple7 and Tuple8

Implement Seq.shuffle()

f1a0ca9

Fix #1318

Merge pull request #1713 from mfejzer/issue1663

3c10a82

Add autogenerated Tuple narrow methods for issue #1663

Change append and prepend in Vector to be Traversable

6b37695

Optimized prependAll and appendAll in Vector using the new size…

3a4339e

… hint

Changed ArrayType methods to favor inlining and branch prediction

070ffe8

Made filter and map smaller in Vector to favor inlining

a6ae577

Unified index related methods in Vector

7203ac9

Add default size to groupBy's LinkedHashMap

0cb69cf

Merge pull request #1715 from paplorinc/append-prepend

adab459

General `Vector` speedup

Added Stream.concat() methods

b698988

Merge pull request #1717 from danieldietrich/stream-concat

c9847f7

Added Stream.concat() methods

apply PR comments

d843319

Merge pull request #1711 from ruslansennov/iss1495

9382f16

introduces Ordered interface #1495

Merge pull request #1712 from io7m/master

f53ff8e

Implement Seq.shuffle()

Traversable.isDistinct() #1635

6776f14

codestyle fixed

3dfe0a9

Fixed Jigsaw args sent to maven compiler in the child POMs

9d9012e

neomatrix369 and others added 8 commits May 19, 2017 22:55

Upgraded to the new maven compiler plugin 3.6.2-SNAPSHOT

3a460b0

Merge pull request #1 from neomatrix369/make-maven-build-using-jigsaw

138e686

Fixed Jigsaw args sent to maven compiler in the child POMs

Prep for merge

0a5de49

Merge branch 'master' of github.com:kittylyst/vavr

de565f5

Still working on MCOMPILER bug

f448659

Move sources to correct level

ca06d20

Update pom.xml

Merge branch 'master' into master

910f9ef

Merge pull request #2 from rfscholte/master

6dbec20

Move sources to correct level

kittylyst and others added 4 commits May 22, 2017 15:43

Upgrade bundle plugin version

c22af62

Use latest Bnd snapshot

cc61ef5

The 3.4.0-SNAPSHOT build will not fail on module-info.class files.

Merge pull request #3 from bjhargrave/patch-1

f29622b

Use latest Bnd snapshot

Unresolved imports have to be removes

16a5531

paplorinc reviewed Jul 14, 2017

View changes

danieldietrich mentioned this pull request Sep 18, 2017

Build failing on JDK 9 / vavr 0.9.1 #2097

Closed

paplorinc suggested changes Sep 20, 2017

View changes

danieldietrich mentioned this pull request Sep 22, 2017

Low impact solution for helping migration to Java 9 modules #2104

Merged

paplorinc suggested changes Sep 23, 2017

View changes

paplorinc mentioned this pull request Sep 23, 2017

Make VAVr Java 8 & 9 compatible #2105

Merged

danieldietrich closed this Sep 29, 2018

danieldietrich force-pushed the master branch from 0e8e6a2 to 4d275ff Compare September 29, 2018 20:40

Initial Spike to support JDK 9 #1994

Initial Spike to support JDK 9 #1994

kittylyst commented May 13, 2017

nfekete commented May 13, 2017

kittylyst commented May 13, 2017

nfekete commented May 13, 2017

kittylyst commented May 14, 2017

danieldietrich commented May 16, 2017

kittylyst commented May 22, 2017

kittylyst commented May 25, 2017

danieldietrich commented Jun 13, 2017

paplorinc Jul 14, 2017

danieldietrich commented Jul 15, 2017

paplorinc left a comment

paplorinc Sep 20, 2017

paplorinc Sep 20, 2017

paplorinc Sep 20, 2017

paplorinc Sep 20, 2017 •

edited

paplorinc Sep 20, 2017

paplorinc Sep 20, 2017

paplorinc Sep 20, 2017

paplorinc Sep 20, 2017

paplorinc Sep 23, 2017

paplorinc Sep 23, 2017

paplorinc Sep 23, 2017

Initial Spike to support JDK 9 #1994

Initial Spike to support JDK 9 #1994

Conversation

kittylyst commented May 13, 2017

nfekete commented May 13, 2017

kittylyst commented May 13, 2017

nfekete commented May 13, 2017

kittylyst commented May 14, 2017

danieldietrich commented May 16, 2017

kittylyst commented May 22, 2017

kittylyst commented May 25, 2017

danieldietrich commented Jun 13, 2017

Choose a reason for hiding this comment

danieldietrich commented Jul 15, 2017

paplorinc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paplorinc Sep 20, 2017 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paplorinc Sep 20, 2017 •

edited