Add random_state parameter

[n-voisine]

Allow Khiops and pykhiops to produce different results by modulating the random seed of Khiops.

For example: measure the variance of the AUC with respect to the automatic generation of aggregates or even the SNB.

[marcboulle]

Variance Analysis, Parameters and Reproducibility via Khiops

Evaluating the Variance of Prediction Results

• This is of natural interest in data mining.
• It is usually addressed using techniques such as cross-validation or bootstrapping, where the analysis method is applied on multiple samples.

Evaluating the Variance of an Algorithm with Respect to Its Parameters

• This process is called sensitivity analysis.
• It is particularly important for new techniques and research purposes.
• It mainly concerns user-defined technical parameters (e.g., the width or shape of a kernel) to provide recommendations for their optimal choices.
• Generally, random seeds are not considered as user parameters, but it can be interesting to study the variance of results with respect to the random seed.
  • High variance related to the random seed usually indicates instability, which is undesirable.

Khiops Parameters

• Khiops does not have such technical parameters
• Khiops is an AutoML solution designed to be as simple as possible to use.
• Only a minimal set of user parameters are available:
  • Problem specification: dictionary, database, target variable.
  • Business constraints:
    • Trade-off between accuracy, modeling, training, and deployment time (e.g., number of generated features, trees...).
    • Interpretability constraints (e.g., maximum number of selected variables, maximum number of part per rpeprocessed variable...).
  • Others as needed.
• Technical parameters are never exposed to the user.
  • They may be available as hidden parameters for the Khiops team, for research purposes.

Reproducibility of Results

• Importance:
  • For the user: ensuring that running the same workflow produces consistent results.
  • For debugging: reproducing issues reported by users.
  • For extensive non-regression testing performed by the Khiops team.
• How:
  • Fixing the random seed for each algorithm (preprocessing, feature engineering, SNB, etc.).
  • Managing seeds in parallelized algorithms to ensure consistent results regardless of the number of cores used.
  • Using stable sorting methods.
  • Fixing advanced compiler options to ensure stability across different operating systems.
  • etc.

[folmos-at-orange]

Maybe this is helpful for your problem @n-voisine : https://khiops.org/api-docs/python-api/api/samples/samples.html#samples.train_predictor_with_cross_validation