Drop function 22 limit #1758

odersky · 2016-11-30T13:36:43Z

Functions with more than 22 parameters are now automatically converted to functions taking
a single object array parameter.

This has been achieved using two tricks:

Create function types of arbitrary arities on demand.
This was achieved by tweaking the scope of the scala package.
Eliminate function types of large arity by mapping them to a new
FunctionXXL type that takes all parameters in a single object array.
This was achieved by tweaking erasure.

Other things I have tried that did not work out well:

Use a single function type in typer. The problem with this
one which could not be circumvented was that existing higher-kinded
code with e.g. Functor assumes that Functon1 is a binary type constructor.
Have a late phase that converts to FunctonXXL instead of
doing it in erasure. The problem with that one was that
potentially every type could be affected, which was ill-suited
to the architecture of a miniphase.

Review by @OlivierBlanvillain ?

In definitions some of the new... methods entered the created symbol while others did not. We now make that distrinction clear in the name.

We know create FunctionN types on demand whenever their name is looked up in the scope of package `scala`. This obviates the need to predefine function traits 23 to 30.

Function classes beyond 22 are now generated on demand, with no upper limit.

Functions with more than 22 parameters are now automatically converted to functions taking a single object array parameter. This has been achieved by tweaking erasure. Other things I have tried that did ot work out well: - Use a single function type in typer. The problem with this one which could not be circumvented was that existing higher-kinded code with e.g. Funcor assumes that Functon1 is a binary type constructor. - Have a late phase that converts to FunctonXXL instead of doing it in erasure. The problem with that one was that potentially every type could be affected, which was ill-suited to the architecture of a miniphase.

odersky · 2016-11-30T14:25:36Z

The PR number #1758 is interesting: 17 + 5 is 22 and 17+5+8 = 30, our previous limit of the arity of temporary functions 😉

sledorze · 2016-11-30T14:27:27Z

fun all way down :)

OlivierBlanvillain

Looks great! I've tried running the full test suite with MaxImplementedFunctionArity = 3 which revealed are quite a few failing test cases, for instance:

def f0(f: (Int, Int, Int) => Int): (Int, Int) => Int =
  f(0, _, _)

I wonder if we could setup the infrastructure to re-run some tests with custom MaxImplementedFunctionArity values, because expending these test cases by hand to arity 23 would be quite tedious...

OlivierBlanvillain · 2016-11-30T14:20:37Z

compiler/src/dotty/tools/dotc/core/Definitions.scala

@@ -646,6 +684,8 @@ class Definitions {
    tp.derivesFrom(NothingClass) || tp.derivesFrom(NullClass)

  def isFunctionClass(cls: Symbol) = isVarArityClass(cls, tpnme.Function)
+  def isUnimplementedFunctionClass(cls: Symbol) =
+    isFunctionClass(cls) && cls.name.functionArity >= MaxImplementedFunctionArity


Should it be strict equality instead?

yes, indeed.

OlivierBlanvillain · 2016-11-30T14:42:33Z

compiler/src/dotty/tools/dotc/core/TypeErasure.scala

+      val sym = tp.symbol
+      sym.isClass &&
+      sym != defn.AnyClass && sym != defn.ArrayClass &&
+      !defn.isUnimplementedFunctionClass(sym)


Should we implement isImplementedFunctionClass instead to avoid the double negation?

It's just one occurrence, so probably not.

OlivierBlanvillain · 2016-11-30T14:50:27Z

library/src/scala/FunctionXXL.scala

+** /____/\___/_/ |_/____/_/ | |                                         **
+**                          |/                                          **
+\*                                                                      */
+// GENERATED CODE: DO NOT EDIT. See scala.Function0 for timestamp.


Copy-past leftover?

OlivierBlanvillain · 2016-11-30T14:50:42Z

compiler/src/dotty/tools/dotc/transform/Erasure.scala

@@ -423,6 +443,9 @@ object Erasure extends TypeTestsCasts{
      }
    }

+	/** Besides notmal typing, this method collects all arguments


Indentation is off

OlivierBlanvillain · 2016-11-30T14:51:39Z

library/src/scala/FunctionXXL.scala

+/** A function with all parameters grouped in an array. */
+trait FunctionXXL {
+
+  def apply(xs: Array[Object]): Object


What's the motivation for Object instead of Any?

FunctionXXL is only used after erasure, when Any is mapped to Object anyway. So I prefer to be explicit.

OlivierBlanvillain · 2016-11-30T15:00:27Z

compiler/src/dotty/tools/dotc/transform/Erasure.scala

@@ -470,18 +499,36 @@ object Erasure extends TypeTestsCasts{
      super.typedValDef(untpd.cpy.ValDef(vdef)(
        tpt = untpd.TypedSplice(TypeTree(sym.info).withPos(vdef.tpt.pos))), sym)

+    /** Besides normal typing, this function also compacts anonymous functions
+     *  with more than `MaxImplementedFunctionArity` parameters to ise a single


s/to ise/into/?

odersky · 2016-11-30T17:47:01Z

Looks great! I've tried running the full test suite with MaxImplementedFunctionArity = 3 which revealed are quite a few failing test cases, for instance:

I should have mentioned: Only the implemented function traits have tupled and curried methods. The synthetic ones lack them.

OlivierBlanvillain · 2016-12-01T13:49:46Z

compiler/src/dotty/tools/dotc/core/Definitions.scala

+  def isFunctionType(tp: Type)(implicit ctx: Context) =
+    isFunctionClass(tp.dealias.typeSymbol) && {
+      val arity = functionArity(tp)
+      arity >= 0 && tp.isRef(FunctionType(functionArity(tp)).typeSymbol)


functionArity(tp) could be replaced with arity

odersky · 2016-12-03T10:01:33Z

I think we should add the testing setup in another PR (if we get around to it). To recall: We'd need to move all curried and tupled functions into decorators. Then we should be able to reduce MaxImplementedFunctionArity at will.

odersky added 4 commits November 29, 2016 19:31

Make clear where symbols are entered or not.

f7001a4

In definitions some of the new... methods entered the created symbol while others did not. We now make that distrinction clear in the name.

Create FunctionN types on demand

e7a0d03

We know create FunctionN types on demand whenever their name is looked up in the scope of package `scala`. This obviates the need to predefine function traits 23 to 30.

Drop limit 30 of generated function classes

3116142

Function classes beyond 22 are now generated on demand, with no upper limit.

odersky added the stat:in progress label Nov 30, 2016

Add missing file

636608a

odersky added stat:needs review and removed stat:in progress labels Nov 30, 2016

OlivierBlanvillain reviewed Nov 30, 2016

View reviewed changes

ecompositor approved these changes Nov 30, 2016

View reviewed changes

Adress reviewers comments

055f12f

OlivierBlanvillain reviewed Dec 1, 2016

View reviewed changes

odersky merged commit 8288a34 into scala:master Dec 3, 2016

nicolasstucki mentioned this pull request Dec 18, 2016

Lambdas with more than 22 parameters crash the compiler. #1384

Closed

allanrenucci deleted the change-functions branch December 14, 2017 16:58

robin-aws mentioned this pull request Jan 17, 2020

Parser should not dynamically inject shared builtin program components dafny-lang/dafny#511

Closed

odersky mentioned this pull request May 12, 2022

Fix documentation regressions #15170

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop function 22 limit #1758

Drop function 22 limit #1758

odersky commented Nov 30, 2016 •

edited by OlivierBlanvillain

Loading

odersky commented Nov 30, 2016

sledorze commented Nov 30, 2016

OlivierBlanvillain left a comment

OlivierBlanvillain Nov 30, 2016

odersky Dec 1, 2016

OlivierBlanvillain Nov 30, 2016

odersky Dec 1, 2016

OlivierBlanvillain Nov 30, 2016

OlivierBlanvillain Nov 30, 2016

OlivierBlanvillain Nov 30, 2016

odersky Dec 1, 2016

OlivierBlanvillain Nov 30, 2016

odersky commented Nov 30, 2016

OlivierBlanvillain Dec 1, 2016

odersky commented Dec 3, 2016

Drop function 22 limit #1758

Drop function 22 limit #1758

Conversation

odersky commented Nov 30, 2016 • edited by OlivierBlanvillain Loading

odersky commented Nov 30, 2016

sledorze commented Nov 30, 2016

OlivierBlanvillain left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

odersky commented Nov 30, 2016

Choose a reason for hiding this comment

odersky commented Dec 3, 2016

odersky commented Nov 30, 2016 •

edited by OlivierBlanvillain

Loading