Summary
ContinuousOrdinalTransformer produces silently wrong output when given a pandas
DataFrame: the wrong number of mappings, the wrong output shape, and all-zero codes. No
exception is raised.
fit iterates for col in X.T, which for a DataFrame yields the column labels of the
transpose — i.e. the original row index — not the column data. transform iterates
for row in X, which for a DataFrame yields column names, not rows.
Reproduction
import numpy as np, pandas as pd
from pretab.transformers import ContinuousOrdinalTransformer
df = pd.DataFrame({"c1": ["a", "b", "a", "c"], "c2": ["x", "y", "x", "y"]})
t = ContinuousOrdinalTransformer().fit(df)
print("len(mapping_):", len(t.mapping_), "(expected 2)")
print("mapping_:", t.mapping_)
print("transform(df):\n", t.transform(df), " <- expected shape (4, 2)")
len(mapping_): 4 (expected 2)
mapping_: [{np.int64(0): 1, None: 0}, {np.int64(1): 1, None: 0}, {np.int64(2): 1, None: 0}, {np.int64(3): 1, None: 0}]
transform(df):
[[0 0]
[0 0]] <- expected shape (4, 2)
mapping_ has one entry per row (keyed by the row index 0..3) instead of one per
column, and transform returns 2 rows of zeros for a 4-row, 2-column frame.
A related shape bug on empty input, which affects ndarray callers too:
import numpy as np
from pretab.transformers import ContinuousOrdinalTransformer
arr = np.array([["a", "x"], ["b", "y"]], dtype=object)
t = ContinuousOrdinalTransformer().fit(arr)
print("transform(empty).shape:", t.transform(np.empty((0, 2), dtype=object)).shape,
"(expected (0, 2))")
transform(empty).shape: (0,) (expected (0, 2))
np.array([]) from the empty list comprehension is 1-D, which breaks any downstream
hstack.
Expected
DataFrame and ndarray inputs behave identically: one mapping per column, output shape
(n_samples, n_features), and (0, n_features) for zero rows.
Actual
DataFrame input yields n_rows mappings and an (n_columns, n_columns) all-zero result;
empty input yields a 1-D array.
Root cause
pretab/transformers/encoders/continuous_ordinal.py:49-52:
self.mapping_ = [
{category: i + 1 for i, category in enumerate(np.unique(col))}
for col in X.T # <- DataFrame.T iterates labels
]
pretab/transformers/encoders/continuous_ordinal.py:73-78:
X_transformed = np.array([
[self.mapping_[col].get(value, 0) for col, value in enumerate(row)]
for row in X # <- DataFrame iterates column names
])
Inside Preprocessor the defect is masked, because the SimpleImputer that precedes this
step in get_categorical_transformer_steps always hands it an ndarray. It bites anyone
using the transformer directly — and it is public, exported from pretab.transformers, and
documented in the API reference.
Suggested fix
Coerce to a 2-D ndarray at the top of both methods and build the output with an explicit
shape:
def fit(self, X, y=None):
X = np.asarray(X, dtype=object)
if X.ndim == 1:
X = X.reshape(-1, 1)
self.mapping_ = [
{category: i + 1 for i, category in enumerate(np.unique(X[:, j]))}
for j in range(X.shape[1])
]
...
def transform(self, X):
check_is_fitted(self, "mapping_")
X = np.asarray(X, dtype=object)
if X.ndim == 1:
X = X.reshape(-1, 1)
out = np.zeros(X.shape, dtype=int)
for j, mapping in enumerate(self.mapping_):
out[:, j] = [mapping.get(v, 0) for v in X[:, j]]
return out
The explicit np.zeros(X.shape, dtype=int) also fixes the empty-input shape for free.
While in there, two adjacent points worth a look:
transform does not validate that X.shape[1] == len(self.mapping_), unlike the other
transformers in the package, which raise PretabDataError via
pretab.core.validation.validate_2d_allow_nan.
- The docstring says "Unknown or missing categories are mapped to 0", and
fit sets
mapping[None] = 0, but NaN is not None — a float("nan") category is treated as an
ordinary category at fit and (usually) as unknown at transform. Worth stating explicitly
or handling.
Impact
Public transformer, exported from pretab.transformers and listed in
docs/api/index.rst. Direct DataFrame use produces plausible-looking but entirely wrong
codes with no error. The Preprocessor path is not affected.
Environment
- pretab 0.1.0 (
main @ 51c3043)
- Python 3.11.15, numpy 2.4.6, pandas 2.3.3, scikit-learn 1.9.0, scipy 1.17.1
- macOS (darwin 25.5.0)
Summary
ContinuousOrdinalTransformerproduces silently wrong output when given a pandasDataFrame: the wrong number of mappings, the wrong output shape, and all-zero codes. No
exception is raised.
fititeratesfor col in X.T, which for a DataFrame yields the column labels of thetranspose — i.e. the original row index — not the column data.
transformiteratesfor row in X, which for a DataFrame yields column names, not rows.Reproduction
mapping_has one entry per row (keyed by the row index0..3) instead of one percolumn, and
transformreturns 2 rows of zeros for a 4-row, 2-column frame.A related shape bug on empty input, which affects ndarray callers too:
np.array([])from the empty list comprehension is 1-D, which breaks any downstreamhstack.Expected
DataFrame and ndarray inputs behave identically: one mapping per column, output shape
(n_samples, n_features), and(0, n_features)for zero rows.Actual
DataFrame input yields
n_rowsmappings and an(n_columns, n_columns)all-zero result;empty input yields a 1-D array.
Root cause
pretab/transformers/encoders/continuous_ordinal.py:49-52:pretab/transformers/encoders/continuous_ordinal.py:73-78:Inside
Preprocessorthe defect is masked, because theSimpleImputerthat precedes thisstep in
get_categorical_transformer_stepsalways hands it an ndarray. It bites anyoneusing the transformer directly — and it is public, exported from
pretab.transformers, anddocumented in the API reference.
Suggested fix
Coerce to a 2-D ndarray at the top of both methods and build the output with an explicit
shape:
The explicit
np.zeros(X.shape, dtype=int)also fixes the empty-input shape for free.While in there, two adjacent points worth a look:
transformdoes not validate thatX.shape[1] == len(self.mapping_), unlike the othertransformers in the package, which raise
PretabDataErrorviapretab.core.validation.validate_2d_allow_nan.fitsetsmapping[None] = 0, butNaNis notNone— afloat("nan")category is treated as anordinary category at fit and (usually) as unknown at transform. Worth stating explicitly
or handling.
Impact
Public transformer, exported from
pretab.transformersand listed indocs/api/index.rst. Direct DataFrame use produces plausible-looking but entirely wrongcodes with no error. The
Preprocessorpath is not affected.Environment
main@ 51c3043)