Named tuples second implementation #19174

odersky · 2023-12-03T13:31:23Z

This implementation follows the alternative representation scheme, where a named tuple type is represented as a
pair of two tuples: one for the names, the other for the values.

Compare with #19075, where named tupes were regular types, with special element types that combine name and value.

In both cases, we use an opaque type alias so that named tuples are represented at runtime by just their values - the names are forgotten.

The new implementation has some advantages

we can control in the type that named and unnamed elements are not mixed,
no element types are leaked to user code,
non-sensical operations such as concatenating a named and an unnamed tuple are statically excluded,
it's generally easier to enforce well-formedness constraints on the type level.

The main disadvantage compared to #19075 is that there is a certain amount of duplication in types and methods between Tuple and NamedTuple. On the other hand, we can make sure by this that no non-sensical tuple operations are accidentally applied to named tuples.

lrytz · 2023-12-05T08:26:14Z

Is it possible to change toString?

scala> println((a = 1, b = 2))
(1,2)

also the REPL doesn't seem to display tuple literals for some reason.

scala> (1, 2)
val res8: (Int, Int) = (1,2)

scala> (a = 1, b = 2)

scala>

library/src/scala/NamedTuple.scala

odersky · 2023-12-05T10:13:55Z

@lrytz We can't change toString since named tuples are to tuples. We should seriously start with implementing a general Show typeclass to address these issues.

I don't know how to run the repl with a language import in the latest release. Who else can look into this?

lrytz · 2023-12-05T10:39:49Z

I don't know how to run the repl with a language import in the latest release. Who else can look into this?

I used

scala> import language.experimental.namedTuples

Another observation:

scala> def f: NamedTuple.NamedTuple[(Int, Any), (Int, String)] = ???
[error] scala.MatchError: TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),class Int) (of class dotty.tools.dotc.core.Types$CachedTypeRef)
[error] 	at dotty.tools.dotc.core.TypeUtils.$anonfun$2(TypeUtils.scala:84)
[error] 	at scala.collection.immutable.List.map(List.scala:246)
[error] 	at dotty.tools.dotc.core.TypeUtils.namedTupleElementTypesUpTo(TypeUtils.scala:84)
[error] 	at dotty.tools.dotc.printing.RefinedPrinter.appliedText$1(RefinedPrinter.scala:241)

noti0na1 · 2023-12-05T16:17:17Z

tests/neg/named-tuples.scala

+  val _: (String, Int) = (name = "", age = 0) // error
+  val _: NameOnly = person // error
+  val _: Person = nameOnly // error
+


val _: Person = (name = "") ++ nameOnly

compiler/src/dotty/tools/dotc/core/TypeUtils.scala

tests/run/named-tuples.scala

smarter · 2023-12-08T17:45:41Z

library/src/scala/NamedTuple.scala

+@experimental
+object NamedTuple:
+
+  opaque type AnyNamedTuple = Any


I guess this separate definition is needed because NamedTuple[?, ?] is illegal, but this definition is very limiting compared to what can be done with a regular Tuple:

def elems(x: Tuple): Int = x.size // ok def wrap(x: Tuple): Tuple.Map[x.type, List] = x.map([t] => (x: t) => List(x)) // ok

None of these work on x: AnyNamedTuple because no extension methods are defined on it.

I can't think of a better way to enable this than writing something like:

opaque type AnyNamedTuple = Tuple extension (x: AnyNamedTuple) inline def toTuple: Tuple = x inline def size: Tuple.Size[Tuple] = toTuple.size // ... inline def map[F[_]](f: [t] => t => F[t]): AnyNamedTuple = toTuple.map(f).asInstanceOf // ...

and hope that the more precise extension is picked when it is usable.

If NamedTuple was covariant in both of its arguments we could maybe write:

opaque type NamedTuple[+N <: Tuple, +V <: Tuple] >: V = V type AnyNamedTuple = NamedTuple[Tuple, Tuple]

But this breaks the reduction of all match types defined in NamedTuple.scala and upper-bounded by AnyNamedTuple for some reason.

Can you try AnyNamedTuple = Tuple? I think I tried that before and it did not work.

That doesn't seem to help, somehow all the variants I tried break reduction:

type Person = (name: String, age: Int) val xx: NamedTuple.Names[Person] = ("name", "age") // error: Found (String, String) Expected: NamedTuple.Names[Person]

Hmm. It works for me.

odersky · 2023-12-14T15:56:57Z

Some new comments on the older thread on subtyping: #19075 (comment)

My recommendation is that we want to be very flexible with tuple convertibility, mirroring what other languages do. So we could have subtyping in one direction and an implicit conversion in the other. Which directions? it comes down to these two questions:

val x: (A, B)
val y1: (a1: A, b1:  B)
val y2: (a2: A, b2: B)
val z1 = if ??? then x else y1 // -- what is the type of z1?
val z2 = if ??? then y1 else y2 // -- what is the type of z2

Subtyping unnamed <:< named with implicit conversion in the other direction answers as follows:

val z1: (a1: A, b1: B)
val z2 // error, untypable [EDIT: Actually, it's AnyNamedTuple]

That is, name info is kept when available, type error when it conflicts.

When swapping the directions, we get:

val z1: (A, B)
val z2: (A, B)

That is, name info is erased unless all parts agree on a name.

Typescript seems to follow the first approach. I also think it makes more sense. I think it's ergonomically better since it detects errors earlier.

soronpo · 2023-12-14T16:30:06Z

val z2 = if ??? then y1 else y2 // -- what is the type of z2

Shouldn't it be a union of both types (a1: A, b1: B) | (a2: A, b2: B) or widened to NonEmptyTuple?

odersky · 2023-12-14T17:53:21Z

Shouldn't it be a union of both types (a1: A, b1: B) | (a2: A, b2: B) or widened to NonEmptyTuple?

Actually, widened to AnyNamedTuple, with my last commit. Before it was a type error but for a weird reason. (we got unreducible inferred type).

odersky · 2024-01-12T11:26:10Z

@sjrd After rebasing to the new match type implementation, I get this:

[error] -- [E191] Type Error: /Users/odersky/workspace/dotty/library/src/scala/NamedTuple.scala:107:36 
[error] 107 |  type Names[X <: AnyNamedTuple] <: Tuple = X match
[error]     |                                    ^
[error]     |                The match type contains an illegal case:
[error]     |                    case NamedTuple.NamedTuple[n, _] => n
[error]     |                (this error can be ignored for now with `-source:3.3`)
[error] 108 |    case NamedTuple[n, _] => n
[error] -- [E191] Type Error: /Users/odersky/workspace/dotty/library/src/scala/NamedTuple.scala:111:41 
[error] 111 |  type DropNames[NT <: AnyNamedTuple] <: Tuple = NT match
[error]     |                                         ^
[error]     |                The match type contains an illegal case:
[error]     |                    case NamedTuple.NamedTuple[_, x] => x
[error]     |                (this error can be ignored for now with `-source:3.3`)
[error] 112 |    case NamedTuple[_, x] => x
[error] two errors found

The error message is not exactly helpful. Can you advise what the problem is and how to fix it?

sjrd · 2024-01-12T11:44:05Z

The definition of the two match types:

  /** The names of a named tuple, represented as a tuple of literal string values. */
  type Names[X <: AnyNamedTuple] <: Tuple = X match
    case NamedTuple[n, _] => n

  /** The value types of a named tuple represented as a regular tuple. */
  type DropNames[NT <: AnyNamedTuple] <: Tuple = NT match
    case NamedTuple[_, x] => x

are in a scope that sees the transparent alias NamedTuple[N, V] = V, i.e., NamedTuple = [N, V] =>> V. Therefore, when it sees

    case NamedTuple[n, _] => n

Since NamedTuple is neither a class type, nor an abstract type constructor, and there is at least one type capture in its type arguments, it is an illegal match type case.

This makes sense because it is equivalent by dealiasing to

    case ([N, V] =>> V)[n, _] => n

and that means it collapses to

    case _ => n // oops! n is not defined here!

The "standard workaround" is to move the definition of these match types to a scope that does not see the transparent alias. This exploits the fact that NamedTuple will be an abstract type constructor in such a scope.

odersky · 2024-01-12T12:56:35Z

Thanks for the explanation! I'll try that. It would be good to work on the error diagnostics so that one could suggest fixes like this automatically.

docs/_docs/reference/experimental/named-tuples.md

dwijnand

Mostly minor questions and comments. I think the only real issue I have is the subtype relationship. The test cases seems satisfied if we only support using tuple syntax to define a named tuple, looking to get a green PR of that in #19532.

compiler/src/dotty/tools/dotc/ast/Desugar.scala

library/src/scala/NamedTuple.scala

dwijnand · 2024-01-25T10:33:44Z

compiler/src/dotty/tools/dotc/core/TypeUtils.scala

          case tp: SingletonType =>
-            if tp.termSymbol == defn.EmptyTupleModule then Some(Nil) else None
+            if tp.termSymbol == defn.EmptyTupleModule then Some(Nil)
+            else if normalize then recur(tp.widen, bound)


Why is this and the SkolemType changes needed?

library/src/scala/Tuple.scala

compiler/src/dotty/tools/dotc/ast/Desugar.scala

library/src/scala/NamedTuple.scala

compiler/src/dotty/tools/dotc/core/Contexts.scala

as is the case for `Concat`

to only require being defined on the element types. Similar to what we have for `type FlatMap`

and refine `type IndexOf` doc

This is for the same reason as we changed `type Head[X <: NonEmptyTuple] = ...` to `type Head[X <: Tuple] = ...` Also, this is no more unsafe than the other operations already defined for all tuples. `drop(1)` for example was always defined, even though `tail` wasn't.

The tuple term level definitions were not being tested before

Keep only the changes we need for making NamedTuple work properly. We still keep operations Contains and Disjoint in Tuple and make Reverse standard. We remove or amend tests that relied on the changes to Tuple. No tests about NamedTuple's are affected.

If we add it as is now, we will *not* be able to add the bound in a future release. It is best to leave it completely undefined. The typer is happy to ignore the case where it does not exist at all.

bishabosha

amazing

odersky force-pushed the named-tuples-2 branch from 764c3b0 to 3b25712 Compare December 3, 2023 13:49

odersky added the needs-minor-release This PR cannot be merged until the next minor release label Dec 3, 2023

odersky mentioned this pull request Dec 3, 2023

Improve Tuple and NamedTuple #19175

Open

odersky marked this pull request as draft December 3, 2023 14:24

odersky mentioned this pull request Dec 3, 2023

Named tuples experimental first implementation #19075

Closed

sjrd reviewed Dec 5, 2023

View reviewed changes

library/src/scala/NamedTuple.scala Outdated Show resolved Hide resolved

noti0na1 requested changes Dec 5, 2023

View reviewed changes

smarter requested changes Dec 8, 2023

View reviewed changes

odersky force-pushed the named-tuples-2 branch from 97757f7 to 11c65aa Compare December 13, 2023 17:53

odersky force-pushed the named-tuples-2 branch from 1db08dc to 8d73b15 Compare December 14, 2023 17:52

odersky force-pushed the named-tuples-2 branch from 8d73b15 to 1d1cc08 Compare January 12, 2024 17:48

odersky mentioned this pull request Jan 14, 2024

SIP-58 - Named Tuples scala/improvement-proposals#72

Open

soronpo reviewed Jan 14, 2024

View reviewed changes

docs/_docs/reference/experimental/named-tuples.md Outdated Show resolved Hide resolved

odersky added this to the 3.5.0 milestone Jan 15, 2024

odersky requested a review from dwijnand January 17, 2024 18:05

odersky self-assigned this Jan 17, 2024

dwijnand reviewed Jan 25, 2024

View reviewed changes

dwijnand mentioned this pull request Jan 31, 2024

[WIP] Named tuples PR-PR #19532

Closed

odersky force-pushed the named-tuples-2 branch 2 times, most recently from 13c0b17 to fea6fff Compare February 7, 2024 17:46

dwijnand reviewed Feb 7, 2024

View reviewed changes

compiler/src/dotty/tools/dotc/core/Contexts.scala Outdated Show resolved Hide resolved

odersky and others added 22 commits May 6, 2024 17:11

More tests

984fe62

Mention restriction against _1, _2, ... as named tuple labels

21bcfef

Address review comments on Tuple.scala

92d22c9

Reject recursive dynamicSelect corner case

1613ee1

Update compiler/src/dotty/tools/dotc/typer/Typer.scala

a04d3a7

Update tests/neg/unselectable-fields.check

37b1bd2

Use consistent naming and drop remaining braces in tuple type ops

3f8f6c6

Add an infix shorthand for Append[X, Y]

9627c08

as is the case for `Concat`

Drop unreachable case from type Zip

8a4162f

Document Concat covariance in 2nd parameter

0ab9e7b

Refine bounds of type Filter predicate

075b7d1

to only require being defined on the element types. Similar to what we have for `type FlatMap`

Do contains runtime operation based on term equality

c58c8c2

Do indexOf runtime operation based on term equality

57b17ac

and refine `type IndexOf` doc

Do filter runtime operation based on a term level predicate

f427ec9

Mark type Append 2nd argument as covariant

94b7c1f

Reorder operations to be same between term and type level

40c6138

Drop braces remaining at term level

03509b8

Fix def indexOf to return the size instead of -1

9047ac3

Revert doing tuple runtime operations based on a term level predicates

579e14a

Make named-tuples-strawman-2.scala a run test

8d6fa37

The tuple term level definitions were not being tested before

import language.experimental.namedTuples in pos/fieldsOf.scala

1e29c4f

odersky force-pushed the named-tuples-2 branch from 9c3cf4e to 1e29c4f Compare May 6, 2024 15:18

odersky and others added 2 commits May 7, 2024 09:47

Revert the addition of type Fields in Selectable.

f80a8dd

If we add it as is now, we will *not* be able to add the bound in a future release. It is best to leave it completely undefined. The typer is happy to ignore the case where it does not exist at all.

sjrd approved these changes May 7, 2024

View reviewed changes

sjrd mentioned this pull request May 7, 2024

Named tuples 3 #20346

Closed

bishabosha approved these changes May 7, 2024

View reviewed changes

odersky merged commit 7e27c4b into scala:main May 7, 2024
19 checks passed

odersky deleted the named-tuples-2 branch May 7, 2024 12:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Named tuples second implementation #19174

Named tuples second implementation #19174

odersky commented Dec 3, 2023 •

edited

lrytz commented Dec 5, 2023

odersky commented Dec 5, 2023

lrytz commented Dec 5, 2023

noti0na1 Dec 5, 2023

smarter Dec 8, 2023

smarter Dec 8, 2023

odersky Dec 8, 2023

smarter Dec 8, 2023

odersky Dec 13, 2023

odersky commented Dec 14, 2023 •

edited

soronpo commented Dec 14, 2023 •

edited

odersky commented Dec 14, 2023 •

edited

odersky commented Jan 12, 2024

sjrd commented Jan 12, 2024 •

edited by dwijnand

odersky commented Jan 12, 2024

dwijnand left a comment

dwijnand Jan 25, 2024

bishabosha left a comment

Named tuples second implementation #19174

Named tuples second implementation #19174

Conversation

odersky commented Dec 3, 2023 • edited

lrytz commented Dec 5, 2023

odersky commented Dec 5, 2023

lrytz commented Dec 5, 2023

noti0na1 Dec 5, 2023

Choose a reason for hiding this comment

smarter Dec 8, 2023

Choose a reason for hiding this comment

smarter Dec 8, 2023

Choose a reason for hiding this comment

odersky Dec 8, 2023

Choose a reason for hiding this comment

smarter Dec 8, 2023

Choose a reason for hiding this comment

odersky Dec 13, 2023

Choose a reason for hiding this comment

odersky commented Dec 14, 2023 • edited

soronpo commented Dec 14, 2023 • edited

odersky commented Dec 14, 2023 • edited

odersky commented Jan 12, 2024

sjrd commented Jan 12, 2024 • edited by dwijnand

odersky commented Jan 12, 2024

dwijnand left a comment

Choose a reason for hiding this comment

dwijnand Jan 25, 2024

Choose a reason for hiding this comment

bishabosha left a comment

Choose a reason for hiding this comment

odersky commented Dec 3, 2023 •

edited

odersky commented Dec 14, 2023 •

edited

soronpo commented Dec 14, 2023 •

edited

odersky commented Dec 14, 2023 •

edited

sjrd commented Jan 12, 2024 •

edited by dwijnand