New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Library modularisation #323

Open
adriaanm opened this Issue Mar 2, 2017 · 11 comments

Comments

Projects
None yet
7 participants
@adriaanm
Member

adriaanm commented Mar 2, 2017

We'd like to reduce the standard library to a small, stable, core, comprising the collections and the other standard types, such as Option, TupleN, Either and Try. The Scala Platform process will provide a vibrant complement of modules that can evolve more quickly. To make room for innovation, we will move parts of the standard library out of scala/scala, as we did in 2.11.

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Mar 2, 2017

Member

Parallel collections: scala/scala#5603

Member

adriaanm commented Mar 2, 2017

Parallel collections: scala/scala#5603

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Mar 2, 2017

Member

Next modules to be carved out: scala/scala#5677

Member

adriaanm commented Mar 2, 2017

Next modules to be carved out: scala/scala#5677

@adriaanm adriaanm modified the milestone: 2.13 Mar 2, 2017

@kjsingh

This comment has been minimized.

Show comment
Hide comment
@kjsingh

kjsingh Mar 12, 2017

Are we going to have independent modular releases? And will there be an overhead given that each module you use has to be imported using maven, sbt etc.?

kjsingh commented Mar 12, 2017

Are we going to have independent modular releases? And will there be an overhead given that each module you use has to be imported using maven, sbt etc.?

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@SethTisue

SethTisue Mar 13, 2017

Member

Are we going to have independent modular releases?

Yes, just as we have had now for a while already for existing modules such as scala-xml, scala-parser-combinators, scala-swing, etc. Allowing for modules to be improved on their own schedules, independent of the Scala release schedule, is one of the main benefits of modularization.

And will there be an overhead given that each module you use has to be imported using maven, sbt etc.?

In short, yes.

Ever since Scala 2.11.0 (some years now), we've offered scala-library-all.jar in an effort to ease that, but afaik nobody uses it. I've literally never seen it used or seen anyone suggest using it — in a book, blog post, chat room, nothing.

I expect we'll drop scala-library-all from 2.13 on the assumption that the Scala Platform will, sooner or later, come up with some rough equivalent that will perhaps get more traction than scala-library-all did. I don't have a specific link handy, but I know I've seen the topic covered in the Scala Platform discussions at contributors.scala-lang.org.

Member

SethTisue commented Mar 13, 2017

Are we going to have independent modular releases?

Yes, just as we have had now for a while already for existing modules such as scala-xml, scala-parser-combinators, scala-swing, etc. Allowing for modules to be improved on their own schedules, independent of the Scala release schedule, is one of the main benefits of modularization.

And will there be an overhead given that each module you use has to be imported using maven, sbt etc.?

In short, yes.

Ever since Scala 2.11.0 (some years now), we've offered scala-library-all.jar in an effort to ease that, but afaik nobody uses it. I've literally never seen it used or seen anyone suggest using it — in a book, blog post, chat room, nothing.

I expect we'll drop scala-library-all from 2.13 on the assumption that the Scala Platform will, sooner or later, come up with some rough equivalent that will perhaps get more traction than scala-library-all did. I don't have a specific link handy, but I know I've seen the topic covered in the Scala Platform discussions at contributors.scala-lang.org.

@SethTisue

This comment has been minimized.

Show comment
Hide comment
@SethTisue

SethTisue Mar 20, 2017

Member

I've literally never seen it used

can't say that anymore: https://github.com/search?q=scala-library-all&type=Code&utf8=✓

but 259 uses GitHub-wide seems (in this case, at least) negligible to me.

Member

SethTisue commented Mar 20, 2017

I've literally never seen it used

can't say that anymore: https://github.com/search?q=scala-library-all&type=Code&utf8=✓

but 259 uses GitHub-wide seems (in this case, at least) negligible to me.

@Blaisorblade

This comment has been minimized.

Show comment
Hide comment
@Blaisorblade

Blaisorblade Apr 28, 2017

but 259 uses GitHub-wide seems (in this case, at least) negligible to me.

The conclusion is probably still true, so sorry for the nitpick, but GitHub code search misses results by design (https://help.github.com/articles/searching-code/#considerations-for-code-search). But this time it shouldn't matter too much—not even the worst limitation of all:

At most, search results can show two fragments from the same file, but there may be more results within the file.

Blaisorblade commented Apr 28, 2017

but 259 uses GitHub-wide seems (in this case, at least) negligible to me.

The conclusion is probably still true, so sorry for the nitpick, but GitHub code search misses results by design (https://help.github.com/articles/searching-code/#considerations-for-code-search). But this time it shouldn't matter too much—not even the worst limitation of all:

At most, search results can show two fragments from the same file, but there may be more results within the file.

@szeiger

This comment has been minimized.

Show comment
Hide comment
@szeiger

szeiger May 18, 2017

Further considerations from this discussion:

  • Collection converters and stream converters should end up in the same module.
  • Which parts of the standard library with dependencies on the JRE are reimplemented by Scala.JS and Scala Native? It would be useful to split the library up along these lines.

szeiger commented May 18, 2017

Further considerations from this discussion:

  • Collection converters and stream converters should end up in the same module.
  • Which parts of the standard library with dependencies on the JRE are reimplemented by Scala.JS and Scala Native? It would be useful to split the library up along these lines.
@sjrd

This comment has been minimized.

Show comment
Hide comment
@sjrd

sjrd May 18, 2017

Member

Which parts of the standard library with dependencies on the JRE are reimplemented by Scala.JS and Scala Native?

https://github.com/scala-js/scala-js/tree/master/scalalib

~/projects/scalajs$ find scalalib/overrides scalalib/overrides-2.12 -name "*.scala"
scalalib/overrides/scala/collection/immutable/RedBlackTree.scala
scalalib/overrides/scala/App.scala
scalalib/overrides/scala/concurrent/ExecutionContext.scala
scalalib/overrides/scala/math/ScalaNumber.scala
scalalib/overrides/scala/Array.scala
scalalib/overrides/scala/runtime/ScalaRunTime.scala
scalalib/overrides/scala/runtime/BoxesRunTime.scala
scalalib/overrides/scala/util/DynamicVariable.scala
scalalib/overrides/scala/util/control/NoStackTrace.scala
scalalib/overrides/scala/Symbol.scala
scalalib/overrides/scala/Enumeration.scala
scalalib/overrides-2.12/scala/collection/immutable/Range.scala
scalalib/overrides-2.12/scala/collection/immutable/NumericRange.scala
scalalib/overrides-2.12/scala/collection/mutable/Buffer.scala
scalalib/overrides-2.12/scala/collection/mutable/ArrayBuilder.scala
scalalib/overrides-2.12/scala/concurrent/ExecutionContext.scala
scalalib/overrides-2.12/scala/package.scala
scalalib/overrides-2.12/scala/reflect/NameTransformer.scala
scalalib/overrides-2.12/scala/reflect/Manifest.scala
scalalib/overrides-2.12/scala/reflect/ClassTag.scala
scalalib/overrides-2.12/scala/compat/Platform.scala
scalalib/overrides-2.12/scala/Console.scala

Only a few of those are actually about reimplementations of the JDK:

  • ScalaRunTime.scala and BoxesRunTime.scala (almost part of the compiler; they're completely rewritten for Scala.js)
  • NoStackTrace.scala and App.scala (depend on Java system properties for weird reasons)
  • Symbol.scala (WeakHashMap not supported in Scala.js)
  • Enumeration.scala (run-time reflection ot supported)
  • NameTransformer.scala (also a Java system property IIRC)
  • Console.scala (System.in is not supported)

It is in fact probably not reasonable to split those out. In fact, most of them cannot, because they're super core, and that's the reason we actually do reimplement them, instead of letting them be unsupported. The only one that could be moved away is Enumeration IMO.

The others are more for extra performance and dce, because the implementation in the stdlib is not very good for the JS platform.

Member

sjrd commented May 18, 2017

Which parts of the standard library with dependencies on the JRE are reimplemented by Scala.JS and Scala Native?

https://github.com/scala-js/scala-js/tree/master/scalalib

~/projects/scalajs$ find scalalib/overrides scalalib/overrides-2.12 -name "*.scala"
scalalib/overrides/scala/collection/immutable/RedBlackTree.scala
scalalib/overrides/scala/App.scala
scalalib/overrides/scala/concurrent/ExecutionContext.scala
scalalib/overrides/scala/math/ScalaNumber.scala
scalalib/overrides/scala/Array.scala
scalalib/overrides/scala/runtime/ScalaRunTime.scala
scalalib/overrides/scala/runtime/BoxesRunTime.scala
scalalib/overrides/scala/util/DynamicVariable.scala
scalalib/overrides/scala/util/control/NoStackTrace.scala
scalalib/overrides/scala/Symbol.scala
scalalib/overrides/scala/Enumeration.scala
scalalib/overrides-2.12/scala/collection/immutable/Range.scala
scalalib/overrides-2.12/scala/collection/immutable/NumericRange.scala
scalalib/overrides-2.12/scala/collection/mutable/Buffer.scala
scalalib/overrides-2.12/scala/collection/mutable/ArrayBuilder.scala
scalalib/overrides-2.12/scala/concurrent/ExecutionContext.scala
scalalib/overrides-2.12/scala/package.scala
scalalib/overrides-2.12/scala/reflect/NameTransformer.scala
scalalib/overrides-2.12/scala/reflect/Manifest.scala
scalalib/overrides-2.12/scala/reflect/ClassTag.scala
scalalib/overrides-2.12/scala/compat/Platform.scala
scalalib/overrides-2.12/scala/Console.scala

Only a few of those are actually about reimplementations of the JDK:

  • ScalaRunTime.scala and BoxesRunTime.scala (almost part of the compiler; they're completely rewritten for Scala.js)
  • NoStackTrace.scala and App.scala (depend on Java system properties for weird reasons)
  • Symbol.scala (WeakHashMap not supported in Scala.js)
  • Enumeration.scala (run-time reflection ot supported)
  • NameTransformer.scala (also a Java system property IIRC)
  • Console.scala (System.in is not supported)

It is in fact probably not reasonable to split those out. In fact, most of them cannot, because they're super core, and that's the reason we actually do reimplement them, instead of letting them be unsupported. The only one that could be moved away is Enumeration IMO.

The others are more for extra performance and dce, because the implementation in the stdlib is not very good for the JS platform.

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Nov 6, 2017

Member

With scala/scala#6164, 2.13.0-M3's scala-library and scala-reflect only depend on the compact1 profile. I'd still like to find time to carve out modules, but I think this is already a pretty exciting prospect, allowing much smaller deployments for Scala 2.13 apps.

Member

adriaanm commented Nov 6, 2017

With scala/scala#6164, 2.13.0-M3's scala-library and scala-reflect only depend on the compact1 profile. I'd still like to find time to carve out modules, but I think this is already a pretty exciting prospect, allowing much smaller deployments for Scala 2.13 apps.

@adriaanm

This comment has been minimized.

Show comment
Hide comment
@adriaanm

adriaanm Nov 6, 2017

Member

Thank you to @avkonst for raising this in scala/bug#10559

Member

adriaanm commented Nov 6, 2017

Thank you to @avkonst for raising this in scala/bug#10559

@jvican

This comment has been minimized.

Show comment
Hide comment
@jvican

jvican Jan 15, 2018

Member

I've opened a thread to discuss the future of scala.sys.process https://contributors.scala-lang.org/t/what-do-we-do-with-scala-sys-process/1473.

Member

jvican commented Jan 15, 2018

I've opened a thread to discuss the future of scala.sys.process https://contributors.scala-lang.org/t/what-do-we-do-with-scala-sys-process/1473.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment