Perfomance with pyirf

[gaia-verna]

Hi,
me and @adonini have created two notebooks adapted from the scripts in pyirf for:

1. generating Sensitivity/IRFs files with pyirf (with the same optimization done for ED DL2 files) from protopipe's DL2 files
2. plotting the performance obtained

The notebooks are ready and they work but we would like to discuss some points:

• obviously pyirf should be added to the protopipe environment
• pointing_az/alt columns used in pyirf are not stored in protopipe DL2 files (at the moment we are hard coding these values)
• is gh_score parameter in pyirf equivalent to gammaness and/or score in protopipe DL2 files? @vuillaut @maxnoe @HealthyPear

[HealthyPear]

Hi,
me and @adonini have created two notebooks adapted from the scripts in pyirf for:

I had already opened issue #73 for this...I linked this issue there in case we open other ones

1. generating Sensitivity/IRFs files with pyirf (with the same optimization done for ED DL2 files) from protopipe's DL2 files

You can open a PR with the modification to the code like a suggest in issue #73 and put the results (plots) in the description of the PR (probably better to retain the old script for early comparisons and give the new one another name like e.g. make_DL3.py)

2. plotting the performance obtained

try to work with @vuillaut for this, he was already working on the code to do nice plots in ctaplot

The notebooks are ready and they works but we would like to discuss some points:

* obviously pyirf should be added to the protopipe environment

of course - add it from the pip section

* `pointing_az/alt` columns used in pyirf are not stored in protopipe DL2 files (at the moment we are hard coding these values)

Yes, protopipe never had this info because it saves already the calculated angular distances from the true source position and from the center of the FoV.

A couple of columns more will not change much, so please add them to the DL2 files like e.g.

run_array_direction = event.mcheader.run_array_direction
pointing_az, pointing_alt = run_array_direction[0], run_array_direction[1]
reco_event["pointing_az"] = pointing_az.to("deg").value
reco_event["pointing_alt"] = pointing_alt.to("deg").value

* is `gh_score` parameter in pyirf equivalent to `gammaness` and/or `score` in protopipe DL2 files? @vuillaut @maxnoe @HealthyPear

gammaness and score in protopipe are different because they are related to the specific ML algorithm used

lately, I am using gammaness but you made me think that maybe the gh_score variable should be only one in a DL2 file so the output of protopipe will need to be updated accordigly, yes...
So for the moment you should use whatever you decided to estimate in you analysis.

for the sake of what enters in pyirf, it doesn't matter since the cuts values and domains are decided by the user and in any case (in pyirf) we should enforce the use of efficiency cuts instead of specific values so we do not have to care about the ML output values domain