Skip to content

Add check for "is primitive or mixed Number column" for statistics in compiler plugin #1476

@Jolanrensen

Description

@Jolanrensen

Relevant in GroupBy as well #1090

All statistics functions accept primitive numbers and mixed primitive numbers. This is checked at runtime, but could also be checked at compile time.

For instance: df.mean { intCol and bigNumberCol } is fine to write; the inferred type will be Number, however, this will fail at runtime, because big numbers are not primitives and thus not supported. If the compiler plugin could provide an error message for such a case, it would improve runtime safety of statistics functions :)

I think we can recognize this by checking if all columns adhere to this function:

/**
 * Returns `true` only if [this] represents an optionally nullable primitive number,
 * (like `Double?`, or `Int`), or a "mixed Number" type: `Number?` or `Number`.
 *
 * We don't check for "subtype of Number" to prevent `BigInteger` etc. to be included, but since columns with
 * mixed primitives are allowed in statistics, we do include `Number?` and `Number`
 */
fun ConeKotlinType.isPrimitiveOrMixedNumber(session: FirSession, errorTypesEqualToAnything: Boolean = false): Boolean =
    this.isPrimitiveNumberOrNullableType ||
            this.equalTypes(
                otherType = session.builtinTypes.numberType.coneType,
                session = session,
                errorTypesEqualToAnything = errorTypesEqualToAnything,
            ) ||
            this.equalTypes(
                otherType = session.builtinTypes.numberType.coneType.withNullability(true, session.typeContext),
                session = session,
                errorTypesEqualToAnything = errorTypesEqualToAnything,
            )

Of course, you could also write columnOf<Number>(1.0, BigDecimal(1.0)). We cannot check for this, but since it doesn't occur that often, it's probably fine.

Metadata

Metadata

Assignees

Labels

Compiler pluginAnything related to the DataFrame Compiler PluginenhancementNew feature or request

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions