I want to translate a code from AutoML to pure Python.

[i12popea]

Hello,

I am trying to do a specific conversion of mljar-supervised library training and prediction to the Xgboost algorithm in pure form using Xgboost and scikit-learn libraries among others without using the mljar-supervised library to create the code myself without using the library. It's basically a reverse engineering problem.

The code to translate is this:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn import metrics
from supervised.automl import AutoML
from imblearn.over_sampling import SMOTE,RandomOverSampler
from sklearn.metrics import accuracy_score, classification_report, f1_score, confusion_matrix

df = pd.read_csv(f'data/periodo_final-v3_2.csv',header=0,index_col=0)

X=df[df.columns[1:-1]]
y=df[df.columns[-1]]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=23,stratify=y )
oversample = SMOTE(sampling_strategy=0.8)
X_train, y_train = oversample.fit_resample(X_train, y_train)

automl = AutoML(results_path=f'AutoML_explain_logloss_0.8_conPrecio', mode="Explain", eval_metric="logloss", ml_task='binary_classification', algorithms=['Xgboost'])
automl.fit(X_train, y_train, cv=5)

y_predictions = automl.predict(X_test)

I know that in "Explain" mode the data from X_train and y_train are split at a ratio of 75% for training in the fit method, but I can't figure out how that data is treated.

It's also hard to understand the feval parameter that self.custom_eval_metric uses. This data affects the variable evals_result.

The fit method is called multiple times and I can't understand when it does this, nor how an array of probabilities generated by xgb.train is then obtained into an array of predictions like the AutoML predict method does.

I'm interested in achieving this conversion because I want to customize the code.

Thanks.

[pplonski]

Hi @i12popea,

I wish there would be such feature for automatic Python script creation for each model trained in AutoML.

It is possible to export the model training and interference into Python without mljar-supervised dependency.

The biggest challenge is to write features pre- and post-processing. Do you have only numeric columns in your data?

Please take a look on how to use cv parameter in fit() https://supervised.mljar.com/api/#supervised.automl.AutoML.fit

The xgboost is using logloss as default metric so no need to worry about feval.

[i12popea]

Thanks for answer so soon.

The columns I use in the dataframe are numeric, most are decimal numbers, others are integers. How can I do these pre and post-processing?

I know that the dataset that I enter in the AutoML fit method does 75/25 training and testing, but I don't know if it's random or the way it processes it.

However, I don't understand how you get an array of predictions [0 0 1 0 1 1 ... 1 0] from an array of probabilities [0.00023 0.67 0.34 ... 0.02 0.34]. What is the procedure to achieve this? I've tried but I can't solve this part.

It is important that I get an equivalence in Python, I need to do a series of tests.

All the best.

[pplonski]

If data is numeric then there is no feature processing.

Data is splited randomly. There should be files with exact index for train and validation sets. Please check in results directory.

There is a method for finding an optimal threshold, please check in the code.

[i12popea]

Thanks, I will do it. If I have any other questions I will let you know.