Issues with Scala Standard Library using -Yexplicit-nulls

[WojciechMazur]

My assumption about distributing the Scala 3.8 stdlib compiled with -Yexplicit-nulls was that it would be mostly invisible for users who don't explicitly enable -Yexplicit-nulls. and any forward binarary/source-breaking changes can be included starting with 3.10. However, there are areas in typer which break this assumption.

Below there is a list of so far discovered problems in the OpenCB affected by this change, none of the examples is using -Yexplicit-nulls

Compiler version

3.8 nightlies

Case 1: max-leuthaeuser/scroll

Open CB logs

class DispatchQuery:
  type CompareFunction = (AnyRef, AnyRef) => Boolean
  private var _sortedWith: Option[CompareFunction] = Option.empty

  def sortedWith(f: PartialFunction[(AnyRef, AnyRef), Boolean]): DispatchQuery = {
    // fails
    _sortedWith = Option { case (a, b) => f.applyOrElse((a, b), _ => false) }
    // workaround
    _sortedWith = Option[CompareFunction] { case (a, b) => f.applyOrElse((a, b), _ => false) }
    this
  }

Output

Compiling project (Scala 3.8.0-RC1-bin-20251019-9ff48b1-NIGHTLY, JVM (21))
[error] ./test.scala:29:28
[error] Missing parameter type
[error] 
[error] I could not infer the type of the parameter x$1
[error] in expanded function:
[error]   x$1 =>
[error]   x$1 match 
[error]     {
[error]       case (a, b) =>
[error]         f.applyOrElse((a, b), _$1 => false)
[error]     }
[error] Expected type for the whole anonymous function:
[error]   ((AnyRef, AnyRef) => Boolean) | Null
[error]     _sortedWith = Option { case (a, b) => f.applyOrElse((a, b), _ => false) }
[error]                            ^
[error] ./test.scala:29:58
[error] Not found: a
[error]     _sortedWith = Option { case (a, b) => f.applyOrElse((a, b), _ => false) }
[error]                                                          ^
[error] ./test.scala:29:61
[error] Not found: b
[error]     _sortedWith = Option { case (a, b) => f.applyOrElse((a, b), _ => false) }
[error]                                                             ^
Error compiling project (Scala 3.8.0-RC1-bin-20251019-9ff48b1-NIGHTLY, JVM (21))

Case 2: scullxbones/pekko-persistence-mongo

Similar to case 1 but simpler. Type hint required
https://github.com/VirtusLab/community-build3/actions/runs/18606867633/job/53059127482

trait Result
def getAll(nameFilter: Option[String => Boolean]): List[Result] = ???
def get(collectionName: String): List[Result] =
    getAll(Option(_.startsWith(collectionName)))

Compiling project (Scala 3.8.0-RC1-bin-20251019-9ff48b1-NIGHTLY, JVM (21))
[error] ./test.scala:4:19
[error] Missing parameter type
[error] 
[error] I could not infer the type of the parameter _$1
[error] in expanded function:
[error]   _$1 => _$1.startsWith(collectionName)
[error] Expected type for the whole anonymous function:
[error]   A | Null
[error] 
[error] where:    A is a type variable
[error]     getAll(Option(_.startsWith(collectionName)))
[error]                   ^

Case 3: kitlangton/neotype

Open CB logs

import scala.quoted.*

given OptionFromExpr[T](using Type[T], FromExpr[T]): FromExpr[Option[T]] with {
  def unapply(x: Expr[Option[T]])(using Quotes) = x match {
    case '{ Option[T](${Expr(y)}) } => Some(Some(y))
    case '{ ${Expr(opt)} : Some[T] } => Some(opt)
    case '{ None } => Some(None)
    case _ => None
  }
}

Workaround: Add explicit type narrowing kitlangton/neotype#355

Output

Compiling project (Scala 3.8.0-RC1-bin-20251019-9ff48b1-NIGHTLY, JVM (21))
[error] ./test.scala:5:29
[error] No given instance of type scala.quoted.FromExpr[T | Null] was found for parameter x$2 of method unapply in object Expr.
[error] I found:
[error] 
[error]     scala.quoted.FromExpr.IntFromExpr[T²]
[error] 
[error] But given instance IntFromExpr in object FromExpr does not match type scala.quoted.FromExpr[T | Null]
[error] 
[error] where:    T  is a type in class OptionFromExpr
[error]           T² is a type variable with constraint <: Int
[error] .
[error]     case '{ Option[T](${Expr(y)}) } => Some(Some(y))
[error]                             ^
[error] ./test.scala:5:50
[error] Not found: y
[error]     case '{ Option[T](${Expr(y)}) } => Some(Some(y))
[error]       

Expectation

To be decided if these kind of issues should be handled by the compiler, or does it require manual adjustment by the downstream users

[noti0na1]

For the first two cases, it is caused by an inference bug related to union type in general. If the type is T | Null (or | Int, or any union type, can trigger the same bug), even we know T is bounded by some function type, we are not able to instantiate T | Null to a nullable function type.

def f[T](x: T): T = ???
def f2[T](x: T | Null): T = ???

def test =
  val g: AnyRef => AnyRef = f {
    x => x  // ok
  }
  val g2: AnyRef => AnyRef = f2 {
    x => x // error
  }

In the third case, the error is expected, but we didn't expect users writing their own OptionFromExpr, instead of using the updated one in stdlib.

For all the errors, I don't a quick and safe solution.

Although T | Null is definitaly a more correct signature for Option.apply (and one of the main motivation why we updated the stdlib), we decide to revert the changes related to Option.apply to avoid any breaking change in regular projects.

cc @sjrd @hamzaremmal @olhotak