Add Scala 2.13.x support #1916

WojciechMazur · 2020-10-05T13:35:05Z

This PR adds support for 2.13 Scala version on top of not yet merged work of Ergys on #1877
Resolves #1629

CI was temporally disabled for this PR but it passed all tests when run locally

Before merging this PR following changes should be addressed:

Merging Add Scala 2.12 support #1877 and rebasing this PR on top of incoming changes
Adjusting CI process to allow less time consuming testing - with current pipeline it may take up to 10 hours of CI uptime (~3-4h real time) even if it would fail on scalafmt check. This may cause DoS for other pipelines in case if there is no one to control CI. Travis allows using stages to describe dependencies between jobs, while current setup runs all jobs even if any of them failed.

Changes include:

Allowing cross-building across project (includes fixing deprecation originated from 2.13 changes)
Adding overrides for 2.13, removal of redundant file overrides for other versions and using overrides dir for common changes
Adding scala-collection-compat sources to scalalib overrides for support of new collection API, eg. java.Stream uses underyling scala.Stream/scala.LazyList
Updating auxlib runtime.Statics to changes introduced in 2.13
Symbol changes compat across 2.13.0 - 2.13.2 and later revisions (changed toString method)
Partially porting Hashtable implementation from Scala.js due to cyclic dependency on Properties with underyling scala.Hashmap and new collections implementation using Properties to tune underlying Vector
Added support for non-Integer match cases which are translated to if-else chain.
Test fixes, eg. HashingSuite for case classes which yields different values since 2.13

ekrich · 2020-10-05T14:50:32Z

Wow, this is great but now we will have to cross build our Scala Native libraries. 😄

I have a couple of questions.

What is the plan for updating the scala-collections-compat? Will it be relatively easy to keep up to date?
Since you ported part of the Scala.js collections across, is there a plan or an Issue that can keep track of which parts have been ported so someone can finish or it can be done a little at a time?

WojciechMazur · 2020-10-05T15:02:34Z

@ekrich I don't think there is any plan for updating this lib but it should be quite easy to keep up to date. Although we also can consider removing it and replacing all usages by some other cross-version safe alternative. For now it looked as easiest way to fix this problem and move with migration.
I don't think there is any issue for that right now. In long term it would be great to use common javalib implementation between SN and Scala.js (at least in most parts)

ekrich · 2020-10-05T15:20:59Z

Right now, I use scala-collections-compat in downstream projects as it supports 0.4.0-M2 and all the other cross versions and platforms. Now I'm wondering if that is going to cause problems.

ekrich · 2020-10-05T15:25:39Z

Also, for collections, I added ScalaOps which allows us to remove the scala.collection.JavaConverters. I used that in java.util.Properties. Not sure it that is used elsewhere in Scala.js.

sjrd · 2020-10-05T15:28:57Z

ScalaOps is used in virtually every file of java.util.* in Scala.js ;)

build.sbt

javalib/src/main/scala/java/lang/AbstractStringBuilder.scala

build.sbt

- fixed eta-exapnds - removing deprecated implicit () - removed early initializers - scala.collections compat

…ince they were removed from main distribution in 2.13)

…lectionConverters

…g LazyList since 2.13

…b dependencies

sjrd

A few drive-by comments to hopefully entirely get rid of scala-collections-compat.

util/src/main/scala/scala/scalanative/util/package.scala

build.sbt

tools/src/main/scala/scala/scalanative/build/LLVM.scala

tools/src/main/scala/scala/scalanative/build/IO.scala

tools/src/main/scala/scala/scalanative/build/Discover.scala

build.sbt

…2.13 overrides

…calalib 2.13 overrides" This reverts commit 4ad48f5

sjrd

Looks great. I only have nitpicks left.

javalib/src/main/scala/java/io/BufferedOutputStream.scala

javalib/src/main/scala/java/math/BigDecimal.scala

javalib/src/main/scala/java/net/PlainSocketImpl.scala

javalib/src/main/scala/java/lang/System.scala

javalib/src/main/scala/java/nio/file/attribute/PosixFilePermissions.scala

javalib/src/main/scala/java/util/stream/WrappedScalaStream.scala

nscplugin/src/main/scala/scala/scalanative/nscplugin/package.scala

project/build.sbt

scalalib/overrides-2.13/scala/package.scala

unit-tests/src/test/scala/scala/HashCodeTest.scala

… Vector override lazy

ruurtjan · 2020-11-12T18:53:36Z

So happy to see this merged into master! Thanks for all the effort :)

Is there anything you can say on when this will be released?

sjrd · 2020-11-12T20:02:47Z

Several weeks if we're lucky.

This reverts commit ce712cf.

LeeTibbert reviewed Oct 6, 2020

View changes

build.sbt Outdated Show resolved Hide resolved

WojciechMazur force-pushed the feature/2.13.x-support branch from 88cf241 to d9710fb Compare Oct 7, 2020

WojciechMazur force-pushed the feature/2.13.x-support branch from d9710fb to 38db5d6 Compare Oct 14, 2020

WojciechMazur marked this pull request as ready for review Oct 14, 2020

WojciechMazur force-pushed the feature/2.13.x-support branch 2 times, most recently from b7f9733 to 521b12a Compare Oct 15, 2020

WojciechMazur mentioned this pull request Oct 22, 2020

Feature - Remove redundant scala overrides #1962

Merged

WojciechMazur force-pushed the feature/2.13.x-support branch 3 times, most recently from 4f07a25 to 5ed7bc7 Compare Nov 6, 2020

sjrd requested changes Nov 9, 2020

View changes

This was referenced Nov 10, 2020

Remove usages of scala.collection.JavaConverters #1990

Merged

Remove usage of JavaConverters from javalib #1992

Merged

WojciechMazur added 12 commits Nov 11, 2020

Initial 2.13 cross compile compat:

8a3db5d

- fixed eta-exapnds - removing deprecated implicit () - removed early initializers - scala.collections compat

Updated ScalaVersions

47733cf

Fix early initializers in NirPlugin

b713318

Use java parallel processing instead of scala parallel collections (s…

aeb5bf5

…ince they were removed from main distribution in 2.13)

Removed posix.time import causing wrong reference in scope

d4f7691

Make javalib 2.13 cross-compilable

bea1f01

Replace deprecated scala.collection.JavaConverters with scala.jdk.Col…

f1a49d9

…lectionConverters

Enable implicitConversions in scalalib

c94f59f

Move common overrides to scalalib/overrides

a0db29c

Reimport missing java.Iterable, fix FilesSuite to work with underylin…

11acf24

…g LazyList since 2.13

Test interface/compiler cross-build compat fixes

46a8ee6

Unit-tests cross-build compat fixes

fed20a6

Remove no longer needed collectionsCompat and parallel-collections li…

86672f6

…b dependencies

sjrd reviewed Nov 11, 2020

View changes

WojciechMazur added 8 commits Nov 11, 2020

Add Predef and scala package optimization used in previous versions

a96d1e4

Remove remaining usages of JavaConverters in tools

7c7d875

Remove laziness of Vector if scala package

a6c00a5

Remove laziness of System.systemProperties, it was fixed by scalalib …

4ad48f5

…2.13 overrides

Fix empty code dumps

3dcbf8e

Don't use view in Show.dump. It does not contains sortBy method in 2.13

9589552

Port 2.13 scala.reflect overrides from scala.js

7ab5ba4

Revert "Remove laziness of System.systemProperties, it was fixed by s…

5d5afa1

…calalib 2.13 overrides" This reverts commit 4ad48f5

sjrd requested changes Nov 12, 2020

View changes

WojciechMazur added 6 commits Nov 12, 2020

Removed no longer need libDependency in project/build.sbt

373b22d

Styling fixes in javalib

a8306f4

Fix checking for scala version in HashCodeTest

cd40d25

Add missing param braces in ju.BaseStream and FileAttribute

d0c5cf2

Remove laziness of ju.System.systemProperties. Set 2.13 scala package…

bbcc160

… Vector override lazy

Explicitly use nsc.Global with Singleton instead of using type alias

70eb2ba

sjrd approved these changes Nov 12, 2020

View changes

sjrd merged commit 252d9b7 into scala-native:master Nov 12, 2020

SethTisue mentioned this pull request Nov 12, 2020

for 1.2.x, drop Scala 2.11 support (including Scala Native) scala/scala-parser-combinators#309

Closed

vicopem pushed a commit to vicopem/scala-native that referenced this pull request Nov 19, 2020

Add Scala 2.13.x support (scala-native#1916)

ce712cf

vicopem added a commit to vicopem/scala-native that referenced this pull request Nov 19, 2020

Revert "Add Scala 2.13.x support (scala-native#1916)"

749f08c

This reverts commit ce712cf.

vicopem added a commit to vicopem/scala-native that referenced this pull request Nov 19, 2020

Revert "Add Scala 2.13.x support (scala-native#1916)"

72133ed

This reverts commit ce712cf.

WojciechMazur mentioned this pull request Dec 4, 2020

Lower memory usage in dump stage #1982

Closed

LeeTibbert mentioned this pull request Jan 8, 2021

Fix #1644: Java I/O now checks fcntl.open result & reports errno; requires PR #1635 #1651

Closed

WojciechMazur deleted the feature/2.13.x-support branch Feb 15, 2021

ekrich pushed a commit to ekrich/scala-native that referenced this pull request May 21, 2021

Add Scala 2.13.x support (scala-native#1916)

d839f19

WojciechMazur added a commit to WojciechMazur/scala-native that referenced this pull request Aug 25, 2021

Add Scala 2.13.x support (scala-native#1916)

94b1c05

Add Scala 2.13.x support #1916

Add Scala 2.13.x support #1916

WojciechMazur commented Oct 5, 2020 •

edited

ekrich commented Oct 5, 2020

WojciechMazur commented Oct 5, 2020

ekrich commented Oct 5, 2020

ekrich commented Oct 5, 2020

sjrd commented Oct 5, 2020

sjrd left a comment

sjrd left a comment

ruurtjan commented Nov 12, 2020

sjrd commented Nov 12, 2020

Add Scala 2.13.x support #1916

Add Scala 2.13.x support #1916

Conversation

WojciechMazur commented Oct 5, 2020 • edited

ekrich commented Oct 5, 2020

WojciechMazur commented Oct 5, 2020

ekrich commented Oct 5, 2020

ekrich commented Oct 5, 2020

sjrd commented Oct 5, 2020

sjrd left a comment

Choose a reason for hiding this comment

sjrd left a comment

Choose a reason for hiding this comment

ruurtjan commented Nov 12, 2020

sjrd commented Nov 12, 2020

WojciechMazur commented Oct 5, 2020 •

edited