Skip to content

[BUG] Classifier labels are never encoded to 0..K-1 — crashes on string labels, silently wrong on non-contiguous integers #409

Description

@ChrisW09

Describe the bug
Class labels are never encoded to 0..K-1 anywhere in the classifier fit path. SklearnBaseClassifier.fit stores self.classes_ = np.unique(y) (deeptab/models/classifier_base.py:294) and predict() maps argmax indices back through classes_ (classifier_base.py:386-388), but the raw y values are passed straight through _FitMixin._build_modelTabularDataModule.preprocess_datatorch.tensor(self.y_train, dtype=torch.long) (deeptab/data/datamodule.py:275-287) and used directly as CrossEntropy class indices / BCE targets. No LabelEncoder, np.unique(..., return_inverse=True), or searchsorted exists in the package.

Three concrete failure modes:

  1. String labels crash — the raw y also reaches preprocessor.fit(X_train, y_train), and pretab's PLE decision-tree binning calls float(y):
    ValueError: could not convert string to float: 'yes'. sklearn classifiers must accept string labels.
  2. Non-contiguous multiclass int labels (e.g. {10, 20, 30}): CE receives target 10 with 3 logits → IndexError: Target 10 is out of bounds on CPU. On MPS the fit completes silently (no bounds check) and the model trains on garbage.
  3. Binary labels other than {0,1} (e.g. {5, 7}): BCEWithLogitsLoss silently trains against targets 5.0/7.0. On perfectly separable data the model reaches 47.75% accuracy vs 100% with {0,1} labels. No warning, no error.

To Reproduce

import numpy as np, pandas as pd
from deeptab.models import MLPClassifier

rng = np.random.RandomState(0)
X = pd.DataFrame({"num1": rng.randn(400)})

# (1) string labels -> ValueError in preprocessor
y = np.where(X["num1"] > 0, "yes", "no")
MLPClassifier().fit(X, y, max_epochs=1)

# (2) non-contiguous multiclass -> IndexError on CPU, silent garbage on MPS
y = rng.choice([10, 20, 30], 400)
MLPClassifier().fit(X, y, max_epochs=1, accelerator="cpu")

# (3) binary {5,7} -> silently broken training
y = np.where(X["num1"] > 0, 7, 5)
clf = MLPClassifier()
clf.fit(X, y, max_epochs=30, batch_size=32, accelerator="cpu")
print((clf.predict(X) == y).mean())   # ~0.48 on perfectly separable data; 1.0 with labels {0,1}

Expected behavior
fit should encode labels to contiguous indices (and pass the encoded y to the preprocessor and datamodule); predict/predict_proba already assume this mapping exists. Arbitrary string/integer labels should round-trip like any sklearn classifier.

Screenshots
n/a

Desktop (please complete the following information):

  • OS: macOS (Darwin 25.5.0, arm64)
  • Python version: 3.11.15
  • deeptab Version: 2.0.0 (main @ 4e6a359)

Additional context
torch 2.9.1, lightning 2.6.5, scikit-learn 1.9.0, numpy 2.4.6. The existing test suite always uses 0..K-1 integer labels, which is why this never surfaces. Case (3) is the most dangerous: silent, and the MPS variant of case (2) is silent too. Found during a systematic bug-hunt of v2.0.0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions