Permalink
Browse files

More consistent Query.length (“COUNT(*)”) semantics:

Counting any multi-column collection is now interpreted as COUNT(1),
i.e. it ignores nullability of the columns. The previous approach of
picking a random column led to inconsistent results.

Note that this changes the semantics of counting only one side of an
outer join, where the previous goal was not to include non-matching
rows in the total (equivalent to counting the discriminator column).
This does not make sense anymore for the new outer join operators with
correct Option types. The new semantics are identical to those of Scala
collections.

The old semantics remain for counts of single columns for now.

New test in CountTest.testTableCount, modified results for q6 in
AggregateTest.testGroupBy to reflect the new semantics. Fixes #1237.
  • Loading branch information...
szeiger committed Aug 14, 2015
1 parent 091d518 commit 7333bd954cc80ab6e4b6586ee5c75454f9c30abd
@@ -98,7 +98,7 @@ class AggregateTest extends AsyncTest[RelationalTestDB] {
} yield (u, t)).groupBy(_._1.id).map {
case (id, q) => (id, q.length, q.map(_._1).length, q.map(_._2).length)
}).to[Set]
db.run(mark("q6", q6.result)).map(_ shouldBe Set((1, 3, 3, 3), (2, 3, 3, 3), (3, 2, 2, 2), (4, 1, 1, 0)))
db.run(mark("q6", q6.result)).map(_ shouldBe Set((1, 3, 3, 3), (2, 3, 3, 3), (3, 2, 2, 2), (4, 1, 1, 1)))
}.flatMap { _ =>
val q7 = ts.groupBy(_.a).map { case (a, ts) =>
(a, ts.map(_.b).sum, ts.map(_.b).min, ts.map(_.b).max, ts.map(_.b).avg)
@@ -89,4 +89,22 @@ class CountTest extends AsyncTest[RelationalTestDB] {
} yield (a.id, b.map(_.data))).length.result.named("outerJoinLength").map(_ shouldBe 3)
)
}
def testTableCount = {
class T(tag: Tag) extends Table[(Long, String, Long, Option[Long], Option[Long])](tag, "TABLECOUNT_T") {
def a = column[Long]("ID")
def b = column[String]("B")
def c = column[Long]("C")
def d = column[Option[Long]]("DISCONTINUED")
def e = column[Option[Long]]("E")
def * = (a, b, c, d, e)
}
val ts = TableQuery[T]
DBIO.seq(
ts.schema.create,
ts += (1L, "a", 1L, None, None),
ts.length.result.map(_ shouldBe 1)
).withPinnedSession
}
}
@@ -17,8 +17,10 @@ class CreateAggregates extends Phase {
logger.debug("Converting aggregation function application", n)
val CollectionType(_, elType @ Type.Structural(StructType(els))) = from.nodeType
val s = new AnonSymbol
val ref = Select(Ref(s) :@ elType, els.head._1) :@ els.head._2
val a = Aggregate(s, from, Apply(f, Seq(ref))(tpe)).infer()
val a = Aggregate(s, from, Apply(f, Seq(f match {
case Library.CountAll => LiteralNode(1)
case _ => Select(Ref(s) :@ elType, els.head._1) :@ els.head._2
}))(tpe)).infer()
logger.debug("Converted aggregation function application", a)
inlineMap(a)
@@ -355,13 +355,8 @@ class QueryInterpreter(db: HeapBackend#Database, params: Any) extends Logging {
val CollectionType(_, elType) = args(0)._1
val coll = args(0)._2.asInstanceOf[Coll]
(elType match {
case ProductType(_) =>
coll.iterator.filter { p =>
val v = p.asInstanceOf[ProductValue].apply(0)
v != null && v != None
}
case _ =>
coll.iterator.filter(v => v != null && v != None)
case ProductType(_) => coll
case _ => coll.iterator.filter(v => v != null && v != None)
}).size
case Library.Database => ""
case Library.Degrees =>
@@ -54,3 +54,14 @@ for more information.
Due to packaging constraints imposed by OSGi, :hikaricpapi:`slick.jdbc.hikaricp.HikariCPJdbcDataSource`
was moved from package ``slick.jdbc`` to ``slick.jdbc.hikaricp``.
Counting Option columns
-----------------------
Counting any multi-column collection with `.length` now ignores nullability of the columns. The previous
approach of picking a random column led to inconsistent results. This is particularly relevant when you
try to count one side of an outer join. Up to Slick 3.0 the goal (although not achieved in all cases due
to a design problem) was not to include non-matching rows in the total (equivalent to counting the
discriminator column only). This does not make sense anymore for the new outer join operators (introduced
in 3.0) with correct `Option` types. The new semantics are identical to those of Scala collections.
Semantics for counts of single columns remain unchanged.

0 comments on commit 7333bd9

Please sign in to comment.