Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to the 2.13 standard library #7019

Merged
merged 88 commits into from Aug 25, 2019

Conversation

@smarter
Copy link
Member

commented Aug 8, 2019

Still to do:

  • Fix all the deprecation warnings 馃槺
  • Get all the tests to pass.
  • Update all the projects of the community build (or temporarily disable some of them if we can't easily update them)
  • Switch to the 2.13 case class hashing scheme: fixes #5732

As soon as this PR is merged we should publish a new nightly and use it as the reference compiler, so that we don't spend too much time with a build that uses both the 2.12 and 2.13 stdlib (it works, but it can be confusing and cumbersome).

@dotty-bot
Copy link

left a comment

Hello, and thank you for opening this PR! 馃帀

All contributors have signed the CLA, thank you! 鉂わ笍

Commit Messages

We want to keep history, but for that to actually be useful we have
some rules on how to format our commit messages (relevant xkcd).

Please stick to these guidelines for commit messages:

  1. Separate subject from body with a blank line
  2. When fixing an issue, start your commit message with Fix #<ISSUE-NBR>:
  3. Limit the subject line to 72 characters
  4. Capitalize the subject line
  5. Do not end the subject line with a period
  6. Use the imperative mood in the subject line ("Add" instead of "Added")
  7. Wrap the body at 80 characters
  8. Use the body to explain what and why vs. how

adapted from https://chris.beams.io/posts/git-commit

Have an awesome day! 鈽锔

@smarter smarter force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 3a317ae to 40c9fee Aug 8, 2019

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 8, 2019

@bishabosha I'll look into the failing test cases, meanwhile, feel free to help with any of the other items in the todo list above.

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 8, 2019

On the CI the community build fails to build every project with:

module not found: ch.epfl.lamp#sbt-dotty;0.3.4-SNAPSHOT

But I can't reproduce that locally. Weird, but this doesn't prevent us from working on it locally at least.

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 8, 2019

@milessabin There's (at least) one compilation error for shapeless on 2.13:

[error] -- [E007] Type Mismatch Error: /home/smarter/opt/dotty/community-build/community-projects/shapeless/core/src/main/scala/shapeless/shapeless.scala:64:51
[error] 64 |      WrappedArray.make[String](summonValuesAsArray[mirror.MirroredElemLabels])
[error]    |                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]    |                                Found:    Array[Any]
[error]    |                                Required: Array[String]
[error] one error found

If you have the time, it'd be great if you could take a look at it.

@smarter smarter force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 40c9fee to f8f8a18 Aug 8, 2019

@bishabosha

This comment has been minimized.

Copy link
Member

commented Aug 9, 2019

now I see why there is 馃槺 for deprecation

@milessabin

This comment has been minimized.

Copy link
Contributor

commented Aug 9, 2019

@smarter you should be good after #7020.

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 9, 2019

@milessabin There's still an error

[error] -- [E007] Type Mismatch Error: /home/smarter/opt/dotty/community-build/community-projects/shapeless/core/src/main/scala/shapeless/shapeless.scala:65:31
[error] 65 |      WrappedArray.make[String](summonValuesAsArray[mirror.MirroredElemLabels, String])
[error]    |      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[error]    |      Found:    scala.collection.mutable.ArraySeq[String]
[error]    |      Required: Seq[String]

This can be solved by adding a .toSeq to get an immutable Seq, but there might be a nicer way. Note that you don't necessarily need to keep compiling against both the 2.12 and 2.13 stdlib, I'm happy with a 2.13-only git branch.

@milessabin

This comment has been minimized.

Copy link
Contributor

commented Aug 9, 2019

@smarter I'll add the .toSeq for now and switch over to 2.13-only once this PR is merged.

@milessabin

This comment has been minimized.

Copy link
Contributor

commented Aug 9, 2019

Done in #7021.

@bishabosha bishabosha force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 5cc7de5 to 0099330 Aug 9, 2019

@smarter
Copy link
Member Author

left a comment

@bishabosha Would be good to add some test cases for the hashcode changes.

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 9, 2019

Would be good to add some test cases for the hashcode changes.

Actually we already have a related test that should be updated: https://github.com/lampepfl/dotty/blob/master/tests/run/caseClassHash.scala https://github.com/lampepfl/dotty/blob/master/tests/run/caseClassHash.check

@bishabosha

This comment has been minimized.

Copy link
Member

commented Aug 9, 2019

with the new update ## and MurmurHash produce the same value

@bishabosha bishabosha force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 1ded229 to abb75a0 Aug 12, 2019

@milessabin

This comment has been minimized.

Copy link
Contributor

commented Aug 14, 2019

@bishabosha what's with the shapeless bump? That's already merged on master ... wouldn't it be easier to rebase?

@bishabosha bishabosha force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 88dfe50 to 44c1a98 Aug 14, 2019

@smarter smarter force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 490fc95 to 4762d30 Aug 14, 2019

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 14, 2019

@milessabin Rebased with your patch in.

@smarter

This comment has been minimized.

Copy link
Member Author

commented Aug 14, 2019

@liufengyun Unfortunately, https://github.com/liufengyun/bench/tree/master/tests/scala-library does not compile anymore now that the compiler runs using the 2.13 stdlib. I tried to hack it to make it compile, but it seems pretty hard, so I think we'll have to remove it from the benchmarks, and replace it by the new stdlib.

@bishabosha bishabosha force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 45c46d2 to fffc3c4 Aug 15, 2019

@smarter smarter force-pushed the dotty-staging:upgrade/scalac-2.13 branch from e0a6fab to be5e67d Aug 16, 2019

smarter added 5 commits Apr 9, 2018
Adapt/remove tests for the 2.13 stdlib.
For the tests that were originally copied from scalac, I also copied the
updated version if one existed, or removed it if they had removed it.

Unfortunately, I also had to remove a bunch of Dotty tests that relied
on details of the 2.12 stdlib that I couldn't easily port to the 2.13
stdlib.

i3518 has to be disabled because using scala.Unit is marked @compileTimeOnly
and dotty crashes on @compileTimeOnly right now.
Start building Dotty against the 2.13 stdlib
This required some surgery in Build.scala: when compiling the
non-bootstrapped compiler, we need the 2.12 stdlib on the JVM and
compiler classpaths, because the current reference compiler cannot be
run using the 2.13 stdlib, on the other hand when _running_ the
non-bootstrapped compiler we need to the 2.13 stdlib on the compiler
classpath to produce a bootstrapped compiler that links against 2.13.

Upgrade the Scala 2 PickleFormat version to 5.2 (which is in fact
identical to 5.0).

Remove no longer needed -Ynew-collections flag and update the
related test files.
Change extension method naming scheme to match 2.13
See scala/scala#7896 and follow-up
scala/scala#7922, we can do both steps at once
since we always run with a bootstrapped dotty-library on the classpath, even
if the compiler itself is not bootstrapped.
Handle s and raw interpolator with 2.13 stdlib
These methods are now macros, so we always need to expand them.
Cross-compile the compiler with the 2.13 stdlib
We still need to support compiling it against the 2.12 stdlib until we
release a new reference compiler because the current reference compiler
cannot use the 2.13 stdlib. This is mostly not a problem, except that
this forces us to keep two versions of WeakHashSet, the same source file
cannot be used for both the 2.12 and 2.13 stdlib since `-=` and `+=` are
final in 2.13.
smarter and others added 10 commits Aug 23, 2019
Remove dead case and update checkfile
productElementName is no longer generated in Desugar after the previous
commit.
Make i3187 a fuzzy test
It doesn't output an error anymore for some reason, but that's fine: we
only care about it not crashing the compiler, which is what fuzzy tests
check.
Inlining: Constant-fold select trees
This fixes a pickling/unpickling difference in sip23-valueof.scala and
makes sense in general.
Move original t5375 to pending
Needs to be replaced by something that doesn't use the parallel
collections, I attempted to do so in
b9ba16e but the result caused
deadlocks, don't have time to investigate right now.
Fix language server tests
I accidentally replaced `"` by `'` in the generated .dotty-ide.json file
in 2ae438d which broke everything.
Fix -Ytest-pickler handling of aliases
We now need to dealias deeply to get tests/run/Course-2002-13.scala and
tests/run/enrich-gentraversable.scala to pass.
Fix unstable compiler output due to using Set
`membersBasedOnFlags` internally uses `memberNames` which returns a Set,
so the order of its elements is unstable. We were using it when
generating static forwarder methods which lead to some idempotency tests
failing (no idea why this didn't manifest itself before). Fixed by
adding a new `sortedMembersBasedOnFlags` which sorts members by names
and signatures (this required adding a somewhat arbitrary lexicographic
ordering to Signature).

@smarter smarter force-pushed the dotty-staging:upgrade/scalac-2.13 branch 2 times, most recently from dfbb87a to 1ba7e75 Aug 24, 2019

smarter and others added 10 commits Oct 1, 2018
Re-fix test failure on Java 11
Originally fixed in f8a746e, lost in
b9ba16e when tests were updated based
on their version in scala/scala.
Fix infinite-loop when using Jar#iterator
So far, Jar was defined as an Iterable implementation that overrides
`foreach` and then define `override def iterator = this.toList.iterator`
That was OK with the 2.12 stdlib, but stackoverflows with the 2.13 one
due to `toList` now being implemented in terms of `iterator`.

Any way, it turns out that we don't really need Jar to be an Iterable,
so the simplest fix was to just remove these methods, and add back a
`toList` for the one usesite that needed it (not very efficient, but not
any worse than before).
Remove hiding of CanBuildFrom params from the IDE/REPL
Not needed with 2.13 where CanBuildFrom is mostly gone.
Force another nightly to be published
To pick up the improvements since the previous one.

@smarter smarter force-pushed the dotty-staging:upgrade/scalac-2.13 branch from 823254e to 6960f8d Aug 25, 2019

Bump reference compiler to the previous commit
And the revert the CI changes from the previous commit which are no
longer needed.
@bishabosha

This comment has been minimized.

Copy link
Member

commented Aug 25, 2019

馃敟馃敟馃敟

@smarter smarter merged commit 38e87b0 into lampepfl:master Aug 25, 2019

1 check passed

continuous-integration/drone/pr Build is passing
Details

@smarter smarter deleted the dotty-staging:upgrade/scalac-2.13 branch Aug 25, 2019

@anatoliykmetyuk anatoliykmetyuk added this to the 0.18 Tech Preview milestone Aug 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can鈥檛 perform that action at this time.