Skip to content

A dimensionless parameter joins ON TRUE with no single-row guard #166

Description

@FBumann

Not primarily a performance issue — filed alongside the SQL performance batch because it was found reading the same statement, and because the failure mode is a silent row multiplication that also blows up the runtime.

The statement

compiler.py::parameter_join:

on = ' AND '.join(f'{alias}.{d} = {coordinate.format(dim=d)}' for d in declaration.dims) or 'TRUE'
return f'LEFT JOIN p_{param} {alias} ON {on}'

For a parameter with no dims, on is TRUE and the statement is LEFT JOIN p_x b ON TRUE — a cross join against the whole parameter table.

That is correct broadcast semantics if p_x has exactly one row. Nothing checks that it does. _create_param_table accepts whatever the source relation contains:

self._con.execute(f'CREATE TABLE p_{p.name} AS SELECT {collist} FROM {rel}')

A two-row scalar source therefore doubles every row of the frame it is joined into. In a bound (compiler.py::bound, alias b_*) that means duplicate cols entries for the same var_label, which the solver ingests as repeated columns. In a where-mask (alias w_*) it duplicates mask rows. Both build and solve without a word.

Why it is worth stopping at load time

The engine's whole error posture is load-time and specific — coordinate containment is checked precisely because a mistyped label would otherwise vanish silently (_check_coordinate_containment, #135). A scalar parameter with two rows is the same class of bug, on a path with no check at all.

The fix

In _create_param_table, when p.dims is empty:

SELECT count(*) FROM p_{name}

and raise DataError unless it is exactly 1, naming the parameter and the count. The message should say what a dimensionless parameter means — one row, one value, broadcast everywhere — since a user who got here probably passed a column they thought was indexed.

Cost is one metadata-level count(*) per scalar parameter, so this is free in practice.

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:engineLowering, IR, relational executor, sinksbug:silentPlausible wrong answer or unearned green light; the user gets no signalengine:duckdbSpecific to the duckdb executor — invalidated if the polars path winsperfWall time, memory, or I/O cost — not a wrong answer

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions