Skip to content

PredictCTLearnModel error with a SimpleCNN model #259

@hugo-varenne

Description

@hugo-varenne

PredictCTLearnModel error with a SimpleCNN model
A error is encountered when implementing a PredictCTLearnModel using a CTLearnModel with a SimpleCNN configuration. The type column is not found for predict data inside the _predict_classification function from ctlearn/tools/predict_model.py

To Reproduce
Steps to reproduce the behavior:

  1. Train a TrainCTLearnModel with a SimpleCNN config
c = Config()
c.TrainCTLearnModel.output_dir = "/home/hugo/TM/ml/trained_model/test1/"
c.TrainCTLearnModel.input_dir_signal = "/home/hugo/TM/data/samples/gamma/train/"
c.TrainCTLearnModel.file_pattern_signal = ["gamma_*.h5"]
c.TrainCTLearnModel.model_type = "SingleCNN"
c.CTLearnModel.attention_mechanism = None
c.CTLearnModel.attention_reduction_ratio = None
c.CTLearnModel.architecture = [
            {"filters": 16, "kernel_size": 3, "number": 1},
            {"filters": 32, "kernel_size": 3, "number": 1},
            {"filters": 32, "kernel_size": 3, "number": 1},
            {"filters": 64, "kernel_size": 3, "number": 1},
            {"filters": 128, "kernel_size": 3, "number": 1},
        ]

model = TrainCTLearnModel(config=c)
model.run()
  1. Run a prediction on the generated and trained model
model = os.path.join(ctlearn_mgr_config.workspace_path, f"trained_model", f"test1", "ctlearn_model.cpk")
input_url = os.path.join(directory, filename)
output_url = os.path.join(ctlearn_mgr_config.workspace_path, "trained_model", "test1", "predict", f"{predict_file}.h5")
ctlearn_model_predict.predict(input_url=input_url, reco=RECO, model=model, output_url=output_url)

Obtained behavior

2025-09-29 11:24:27,909 ERROR [ctlearn.ctlearn-predict-mono-model] (tool.run): Caught unexpected exception: 'type' Traceback (most recent call last): File "/home/hugo/miniforge3/envs/ctlearn/lib/python3.10/site-packages/ctapipe/core/tool.py", line 445, in run self.start() File "/home/hugo/miniforge3/envs/ctlearn/lib/python3.10/site-packages/ctlearn/tools/predict_model.py", line 1146, in start super()._predict_classification(example_identifiers) File "/home/hugo/miniforge3/envs/ctlearn/lib/python3.10/site-packages/ctlearn/tools/predict_model.py", line 572, in _predict_classification predict_data["type"].T[1], name=f"{self.prefix}_tel_prediction" File "/home/hugo/miniforge3/envs/ctlearn/lib/python3.10/site-packages/astropy/table/table.py", line 2093, in __getitem__ return self.columns[item] File "/home/hugo/miniforge3/envs/ctlearn/lib/python3.10/site-packages/astropy/table/table.py", line 267, in __getitem__ return OrderedDict.__getitem__(self, item) KeyError: 'type'

Expected behavior
Predict correctly without errors the events in the .h5 file and create predictions files

Additionnal information
The error appears because when using a SingleCNN and not a ResNet (default), multiple softmax layer can be created. The name of the last layer (softmax) changes from the classic "softmax" with ResNet to "softmax_{number of softmax layer}. This behaviour provokes an error with this line when using a type model (only model with a softmax) in the _predict_with_model function :

        prediction_colname = (
            model.layers[-1].name if model.layers[-1].name != "softmax" else "type"
        )

Because the name is not softmax, the prediction_colname isn't attributed with the value "type". This will trigger an error later in the code when searching for a column with the name "type" (not attributed because of the previous error) in the _predict_classification function.

       classification_table.add_column(
            predict_data["type"].T[1], name=f"{self.prefix}_tel_prediction"
        )

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions