Fix DL1 EventSource #1546
Fix DL1 EventSource #1546
Conversation
- Dl1 files contain a boolean mask, we want a list of tel_ids - This is now done with the `tel_mask_to_tel_ids` method
- Only calibrate events if no images are present - Crash if file contains no images or raw data - Iterate dl1 telescopes, not raw event. This is needed if the file contains no raw data.
- In line with the global replacement of `mc` to `simulation`
- Enables one to calculate DL1B data from DL1A files. The saved cleaning mask is applied, specifying a cleaning has no effect right now
- Muon reconstruction and stage1 work with dl1a data now - simulation_configs changed to simulation_config for now to match the SimtelEventSource
- Use the HDF5TableReader to retrieve the obs_id as well
- Setting the scope to module avoids recalculation of the same files - Using tmpdir, its not necessary to use with() statements to create temporary files
- This only worked by accident before - Images default to None in the container, so this checks if information was read
Codecov Report
@@ Coverage Diff @@
## master #1546 +/- ##
==========================================
+ Coverage 90.62% 90.78% +0.16%
==========================================
Files 192 192
Lines 13840 13897 +57
==========================================
+ Hits 12542 12616 +74
+ Misses 1298 1281 -17
Continue to review full report at Codecov.
|
|
This is marked for 0.10.1, but also says "API change". I see only that the mc_header attribute changed to simulation_config (as it should, this was missed in the previous release). I think that is less of an API change than a API bugfix, since the API for EventSources already had that attribute. Are there other larger API changes here? Or can we remove that tag, so it can go in 0.10.1 vs 0.11.0? |
I think thats all, so API bugfix might be more accurate. |
- from_config is not needed, if the eventsource uses the baseclass constructor properly
There was a problem hiding this comment.
Looks good.
One minor comment: the self.datalevels property is accessed several times in the event loop, so it may be better either as a lazyproperty, or better yet to generate the values in the constructor and just return self._datalevels in the property. The latter may be preferred as right now if you call close(), accessing the datalevels property causes an exception since _h5file is no longer open.
Otherwise, once the conflicts are fixed, this looks ready.
- Datalevels are checked multiple times when generating events, this avoids having to call this code every time - Datalevels are remmebered even if the file is closed
Small changes to tools and bugfixes in order to work with the DL1EventSource
SubarrayDescriptionDL1EventSource: Every second event was skipped, because the iterator was advanced twice. That's a huge bug if it didn't work for some random other reason beforedl1eventsource.mc_headerstodl1eventsource.simulation_configin line with the changes to theSimtelEventSourceand container structure changes. To make this behave the same way theSimtelEventSourcedoes when using e.g. thestage-1tool, this returns a single container if the file contains only one simulation run. This makes it somewhat inconsistent for merged files, but I think there a a lot of things to clean up anyway for us to fully support merged dl1 files consistently.