Dummyclassifier with binary and multiclass and with predict proba

[petr-ponomarenko]

Describe the workflow you want to enable

There is case of mixing multiple classifiers with SelectFirstClassifier. This allows to include dummy constant classifiers for specific features conditions. For example this allows to provide fixed (rules based) prediction for some combination of features and their thresholds. Unfortunately to export pipeline with such scenario requires all classifiers to have the same output shape and structure. There are workarounds, but much better us to have dummy classifier with constant multiclass output with probabilities set with parameters.

Describe your proposed solution

Add support of binary and multiclass predictions with parameters controlling predict proba output.

Describe alternatives you've considered, if relevant

There is a way to add dummy records into training data and use real multiclass classifiers in combination with feature filters/droppers and selectfirstclassifier. This approach is not very stable and does not provide that much controll as defining exactly output probabilities.

There might be other ways to do controlled multyclass output for specific feature/thresholds combination in addition to main classifiers of pipeline and export it as pmml model. Please let me know about it. Thank you.

Additional context

Maybe ability to controll output probabilities with input features using function transformers with callables might be even better and broader use case solution.

[jnothman]

Please clarify by providing an example of code you wish would work.

[petr-ponomarenko]

Sure @jnothman . My code looks like this:

#### binary classification with label = 0 or 1

#### some float column to create simple example rule 
#### like for "age < 21.0 label = 1 probability is 100%" :
rule_column = "age"

#### data prep:
n = len(df)
df = df.append(df[0:2])
df[n][rule_column] = 10.0
df[n]["label"] = 1
df[n+1][rule_column] = 30.0
df[n]["label"] = 0

#### pipeline:
mapper = DataFrameMapper(
    #### making sure rule_column is X[0]
    [([rule_column], [ContinuousDomain(with_data = False, missing_values = -9999.)])] + 
    [
    #### all other columns/transformations here
    ]
)
classifier = LGBMClassifier(objective = "binary")
dummy_classifier = PMMLPipeline([("drop", ColumnTransformer([("0","passthrough",[0])])),
                                                            ("dummytree", DecisionTreeClassifier())
])

pipeline = PMMLPipeline([
    ("mapper", mapper),
    ("selector", SelectFirstClassifier([('dummyclassifier', dummy_classifier, "X[0] < 21.0"),
                                                            ('classifier', classifier, "X[0]>-20000")])
    )
])

#### fit and pack for serving:
pipeline.fit(df, df["label"])
sklearn2pmml(pipeline,"model.pmml")

What I want to do instead with control of output structure and values of DummyClassifier is something like to avoid adding dummy records to df and to control predicted probabilities output and classes:

pipeline = PMMLPipeline([
    ("mapper", mapper),
    ("selector", SelectFirstClassifier([('dummyclassifier', DummyClassifier(classes=[0,1], proba=[0.0, 1.0]), "X[0] < 21.0"),
                                                            ('classifier', classifier, "X[0]>-20000")])
    )
])

[glemaitre]

I am still not sure to understand. Do you want a way to specify probability with the strategy="constant" with something like:

dummy = DummyClassifier(strategy="constant", constant={0: 0.1, 1: 0.3, 2: 0.6})
# dummy fit just to get the classes_ attribute
dummy.fit([[0], [1], [2]], [0, 1, 2])
# prediction
dummy.predict_proba([[0]])  # return [[0.1, 0.3, 0.6]] (predict the constant passed)
dummy.predict([[0]])  # return [2] (predict argmax of predict_proba)

[petr-ponomarenko]

Yes exactly. I want DummyClassifier to have ability to have multiclass constant predictions with constant probabilities controlled by parameter.

[glemaitre]

I don't see a big burden regarding the maintenance of such new feature. I assume that we could support of a dict of probabilities.
@jnothman do you see something that I overlooked?