Skip to content

Add Python 3.12 & TF 2.16–2.18 support and fix preprocessing/tuning bugs#28

Merged
edi-iee merged 14 commits into
mainfrom
develop
Jun 11, 2026
Merged

Add Python 3.12 & TF 2.16–2.18 support and fix preprocessing/tuning bugs#28
edi-iee merged 14 commits into
mainfrom
develop

Conversation

@roelofsc

@roelofsc roelofsc commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Enable Python 3.12 support and TensorFlow 2.16–2.18 compatibility (for python 3.11 and python 3.12)

  • Replaced all from tensorflow.keras... imports to from keras... imports for compatibility with both Keras 2.15 and Keras 3.x
  • autoencoder.__call__: Convert all inputs to tf.Tensor before passing to model (Keras 3 no longer allows mixed tensor/non-tensor inputs)
  • Renamed the submodule energy_fault_detector.quick_fault_detection.quick_fault_detector to pipeline so we do not shadow the submodule name with the function name anymore. Also updated the tests so they run in python 3.10 as well.

Prevent conditional features (for conditional autoencoder variants) from being dropped during preprocessing:

  • The ColumnSelector or LowUniqueValueFilter now have a protected_features argument in the fit method, so these components know which features should not be dropped. The FaultDetector now passes the conditonal features as protected_features to the DataPreprocessor.
  • After fitting the DataPreprocessor raises an error if (one of) the protected_features was dropped somehow.
  • Added protect_conditional_features config option (under train, default False for backward compatibility). When set to True, conditional features are protected from being dropped by the preprocessing pipeline.

Handling of missing conditional features:

  • Previously, missing conditional features in sensor_data would cause the training to fail.
  • FaultDetector.fit() now resolves conditional features against the available columns in sensor_data before training. Missing features are logged as warnings and excluded.
  • If all conditional features are unavailable:
    • ConditionalAE falls back to MultilayerAutoencoder (with the same architecture params from config).
    • Sequence models simply clear their conditional_features.
  • If protect_conditional_features=False, a post-preprocessing check detects conditionals dropped by the pipeline and triggers the same fallback logic.
  • At predict time, missing conditional features raise a ValueError (since the trained model requires them).

CI:

  • Use a matrix strategy to run all unittests across Python 3.10, 3.11, and 3.12 (and supported TensorFlow versions).

Bugfixes

  • Model tuning:
    • When tuning, the model did not train for the expected number of 'tune_epochs'. This was fixed by not passing the (stale) EarlyStopping callback.
    • The model weights were potentially reset due to a clear_session call in FaultDetector.tune after loading the model.
  • Prevent the RMSEScore.transform method mutating the original input data when scale=True.

roelofsc added 12 commits June 1, 2026 12:53
…ssing by the `ColumnSelector` or `LowUniqueValueFilter` by adding a protected_features parameter to the fit method.
…rom fitting and do not clear session in FaultDetector.tune after loading the model.
…pipeline.py to prevent shadowing the module in the root __init__.py file. Also updates the CLI tests to patch the objects directly.
…, so we do not have any breaking changes.

- Introduce a flag in the config `protect_conditional_features` to set to True if the user wants to protect these features.
- Update the conditional feature list of the autoencoder if some of the conditions are missing.
@roelofsc
roelofsc requested a review from edi-iee June 9, 2026 16:19
Comment thread energy_fault_detector/anomaly_scores/rmse_score.py
Comment thread energy_fault_detector/data_preprocessing/column_selector.py
Comment thread energy_fault_detector/data_preprocessing/column_selector.py
Comment thread energy_fault_detector/data_preprocessing/data_preprocessor.py
Comment thread energy_fault_detector/fault_detector.py
Comment thread energy_fault_detector/fault_detector.py
@edi-iee
edi-iee merged commit 4fa9b29 into main Jun 11, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants