Skip to content

cols.vtype is a VARCHAR repeated once per column where an ENUM would do #170

Description

@FBumann

Measured in #153, which is being closed and re-cut one change per PR. This is the change; the number below came from that branch.

The statement

executor.py::build:

CREATE TABLE cols (col BIGINT, lb DOUBLE, ub DOUBLE, vtype VARCHAR)

vtype is one of three literals — continuous, binary, integer — written once per column. As VARCHAR it is the widest thing on the row, and the row count is the model's column count.

An ENUM stores the tag instead. Every vtype = '...' comparison in the sinks keeps working unchanged, so nothing downstream moves:

types = ', '.join(f"'{t}'" for t in get_args(plan.VariableType))
self._con.execute(f'CREATE TYPE variable_type AS ENUM ({types})')
self._con.execute('CREATE TABLE cols (col BIGINT, lb DOUBLE, ub DOUBLE, vtype variable_type)')

Deriving the members from plan.VariableType rather than typing them out is the part worth keeping: the type is the single declaration, and a fourth variable type cannot land without the table following it.

Measured

0.87s -> 0.48s writing cols at 10M columns.

Scope

One statement plus the type declaration. No sink changes, no test changes beyond what already covers MILP round-trips (tests/test_milp.py).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:engineLowering, IR, relational executor, sinksengine:duckdbSpecific to the duckdb executor — invalidated if the polars path winsperfWall time, memory, or I/O cost — not a wrong answer

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions