perf(preprocessor): transform runs every pipeline twice by default

[ChrisW09]

Summary

Preprocessor.transform(..., return_array=False) — the default — transforms the data once
through the ColumnTransformer, then re-runs each per-feature pipeline a second time
purely to measure how many columns it produced. The widths are already available on the
fitted ColumnTransformer, and the class's own output_dims_ property uses them.

Roughly 1.9x the necessary work on the default code path.

Reproduction

import time
import numpy as np, pandas as pd
from pretab import Preprocessor

rng = np.random.default_rng(0)
df = pd.DataFrame({f"x{i}": rng.normal(size=20_000) for i in range(20)})
pre = Preprocessor(numerical_method="ple").fit(df, rng.normal(size=20_000))
for label, kw in (("return_array=True ", {"return_array": True}),
                  ("return_array=False", {"return_array": False})):
    pre.transform(df, **kw)                      # warm up
    t0 = time.perf_counter()
    for _ in range(3):
        pre.transform(df, **kw)
    print(f"{label}: {(time.perf_counter() - t0) / 3 * 1000:6.1f} ms")

return_array=True :   23.0 ms
return_array=False:   43.5 ms

Both calls return the same underlying numbers; the dict form just slices the array into
per-feature blocks.

Expected

Splitting the stacked array into per-feature blocks is pure bookkeeping and should cost
approximately nothing on top of the single transform.

Actual

Every pipeline runs twice, so the dict form costs about double.

Root cause

pretab/preprocessor.py:460-469:

transformed_dict = {}
start = 0
for name, transformer, columns in self.column_transformer_.transformers_:
    if transformer == "drop":
        continue
    if hasattr(transformer, "transform"):
        width = transformer.transform(X[columns]).shape[1]     # <- second full transform
    else:
        width = 1
    transformed_dict[name] = transformed_X[:, start : start + width]
    start += width

self.column_transformer_.output_indices_ already maps each transformer name to its exact
output slice. output_dims_ (pretab/preprocessor.py:556-572) reads it correctly.

Suggested fix

Slice directly with the recorded indices:

transformed_dict = {}
for name, _transformer, _columns in self.column_transformer_.transformers_:
    span = self.column_transformer_.output_indices_[name]
    if span.stop - span.start == 0:
        continue
    transformed_dict[name] = transformed_X[:, span]

Besides removing the duplicate work, output_indices_ is exact, whereas the current
width = 1 fallback for a non-estimator entry is a guess. (In practice every entry the
Preprocessor builds is a Pipeline, and on scikit-learn 1.9 an empty remainder is
omitted from transformers_ altogether, so that fallback is not currently reachable — but
the indices are the authoritative source either way.)

A test asserting set(transform(X).keys()) and each block's width against
output_dims_ / get_feature_names_out() would lock the behaviour in.

Impact

Performance only — the returned values are correct today. Affects the default
transform / fit_transform path, so it is on every user's hot loop.

Environment

• pretab 0.1.0 (main @ 51c3043)
• Python 3.11.15, numpy 2.4.6, pandas 2.3.3, scikit-learn 1.9.0, scipy 1.17.1
• macOS (darwin 25.5.0)