Problem
Page._columns decides whether a column holds nested-model cells by checking only col[0]. For Optional[NestedModel] fields — common across Market, Event, Series, Settlement — col[0] is often None when the first response row happens to omit that field, so the per-cell model_dump branch is skipped. Later rows that DO carry a populated nested model are then handed to pandas/polars as raw BaseModel instances.
pandas tolerates this (cells of dtype object), but polars from_records inference fails the Struct dtype the docstring promises (#264): markets_all().to_polars() either raises or yields an object column instead of Struct. The #264 bench harness only covers pages whose first row populates every nested field, so the regression is invisible to existing benchmarks.
Evidence
kalshi/models/common.py:81-88:
fields = type(self.items[0]).model_fields.keys()
columns: dict[str, list[object]] = {}
for field in fields:
col: list[object] = [getattr(item, field) for item in self.items]
if col and isinstance(col[0], BaseModel):
col = [v.model_dump(mode="python") if isinstance(v, BaseModel) else v for v in col]
columns[field] = col
return columns
Only col[0] is probed; if it's None, the dump pass is skipped for the whole column.
Suggested fix
Probe the whole column for the first non-None BaseModel and apply the dump branch when one exists; keep the inner isinstance(v, BaseModel) guard inside the comprehension so None cells stay None for nullable-Struct support:
first_model = next((v for v in col if isinstance(v, BaseModel)), None)
if first_model is not None:
col = [v.model_dump(mode="python") if isinstance(v, BaseModel) else v for v in col]
Add a regression test in tests/test_models.py constructing a Page with two heterogeneously-populated rows (None first, populated second) and asserting the second cell is a dict after _columns() (and that to_polars() yields Struct).
Source
Round-3 independent audit (reviewer: performance).
Problem
Page._columnsdecides whether a column holds nested-model cells by checking onlycol[0]. ForOptional[NestedModel]fields — common acrossMarket,Event,Series,Settlement—col[0]is oftenNonewhen the first response row happens to omit that field, so the per-cellmodel_dumpbranch is skipped. Later rows that DO carry a populated nested model are then handed to pandas/polars as rawBaseModelinstances.pandas tolerates this (cells of dtype
object), but polarsfrom_recordsinference fails theStructdtype the docstring promises (#264):markets_all().to_polars()either raises or yields anobjectcolumn instead ofStruct. The #264 bench harness only covers pages whose first row populates every nested field, so the regression is invisible to existing benchmarks.Evidence
kalshi/models/common.py:81-88:Only
col[0]is probed; if it'sNone, the dump pass is skipped for the whole column.Suggested fix
Probe the whole column for the first non-
NoneBaseModeland apply the dump branch when one exists; keep the innerisinstance(v, BaseModel)guard inside the comprehension soNonecells stayNonefor nullable-Structsupport:Add a regression test in
tests/test_models.pyconstructing aPagewith two heterogeneously-populated rows (Nonefirst, populated second) and asserting the second cell is adictafter_columns()(and thatto_polars()yieldsStruct).Source
Round-3 independent audit (reviewer:
performance).