Multi-class classification implementation

[MortenHolmRep]

Implementation of multi-class classification predictions as per #111 , where the new operational structure would imply
Raw Data -> [noise, muon, neutrino]

In the current implementation, this could also easily be expanded to
Raw Data -> [noise, muon, nu_e, nu_mu, nu_tau]
But a two-step classification, first on [noise, muon, neutrino] and then a second classification, with a neutrino-specific task, training with the identified neutrinos on [nu_e, nu_mu, nu_tau] is probably preferred for neutrino classifications.
comments on an approach or alteration to this multi-class classification implementation are welcome.

Training with the task using examples/train_model.py works with the following alterations, where the target are pid:

task = MulticlassificationTask(
        hidden_size=gnn.nb_outputs,
        target_labels=config["target"],
        loss_function=MultiClassificationCrossEntropyLoss(),
    )

and for the results

results = get_predictions(
        trainer,
        model,
        validation_dataloader,
        [config["target"] + "_noise_pred", config["target"] + "_muon_pred", config["target"]+ "_neutrino_pred"],
        additional_attributes=[config["target"], "event_no"],
    )

Implementation by @Peterandresen12 and I

[Peterandresen12]

Looks good to me :-)

[asogaard]

Hi @MortenHolmRep and @Peterandresen12,

Thanks for this PR! It would make a nice addition to the repo. 🙂

I have added a few suggestions and comments that center on the fact that, currently, the proposed task and loss function hard-code assumption about the number of classes, and the PIDs of the particles being classified. We should try to be more general if we want to implement generic multi-class classification.

[MortenHolmRep]

Hi @MortenHolmRep and @Peterandresen12,

Thanks for this PR! It would make a nice addition to the repo. 🙂

I have added a few suggestions and comments that center on the fact that, currently, the proposed task and loss function hard-code assumption about the number of classes, and the PIDs of the particles being classified. We should try to be more general if we want to implement generic multi-class classification.

I will look into a rework tonight, based on the suggestions 👍

[asogaard]

Hi @MortenHolmRep,

Thanks for the quick iteration! :) I have added comments to a few threads. Please let me know if there's anything else you'd like to discuss there. Otherwise, feel free to re-request a review once you think the code in the PR is ready for a second look. :)

[MortenHolmRep]

Thanks for your review @asogaard! I have added some comments for now. The suggestion you made I'll have to test later today.

[asogaard]

@asogaard Can you review the PR again? I have attempted to clean the code, config and unused packages. The generic classification now works and produces good results. The build fails for the LogCMK class in loss_functions, which is unrelated to my implementation, I do not know why I am getting this error.

Regarding the failing unit test: As far as I can tell, the fact that we change the assertion logic regarding transform_target/transform_inference will affect (some of) the unit test(s) that check for this. This is perfectly OK — we will just have to update the unit tests to reflect this new logic. :) I can have a look at the unit test results once you've had a chance to look at the comments above.

[MortenHolmRep]

I have added a script called "train_classification_model_without_configs.py" that follows the old structure and a new script called "train_classification_model.py" that should adhere to the new structure.
The new setup with configs is not done as I need input on how to set it up properly.

[asogaard]

Hi @MortenHolmRep,

I think this looks great! If it runs as expected, I am all for merging this. There is still the problem of the failing unit test that needs to be resolved. I think it is just a matter of deleting this block as it no longer represents expected behaviour. :)

[Peterandresen12]

Well done!