Add RNO-G data reader base on Mattak and a noise importer which utilises this data reader #519
Conversation
|
Arg, this PR is based on #508. Once this is merged, I can make a rebase and the additional diff disappears... |
|
This should be ready now. The main concerned I had regarding the noiseImporter are fixed |
|
@shallmann I would like to merge #508 before that one. |
…noiseImporter module for RNO-G
…me and trace_start_time
… event index to get a specific event rather than only looping over all. Started to implement some function to get event header information
… list of noise folders
…or function (i.e., run()) and function to return a specific event (read_event). Some fixes
…performance when using the generator (i.e., the run() method) and in particular the uproot backend. Add some logger info to the noiseImporter
…election based on time. Change interface of noiseImporter to accept dict for reader arguments
|
I guess the failing of the unit tests has to do with the desy server which provides the antenna files, can someone confirm? |
|
Should be fixed now, so I've restarted the tests |
There was a problem hiding this comment.
Last couple of comments, hopefully
- while I'm in favour of only having a single reader module for RNO-G data,
mattakis currently not yet pip installable, and does not have a fully functioninguprootbackend yet (as far as I understand). Is it worth keeping one or both old reader modules with a deprecation warning until this has been implemented? I find it hard to judge how much of an additional challenge the installation of mattak + ROOT would present to, e.g., a starting master's or bachelor's student. If the consensus is it's doable, I'm also happy to only keep the new one. - The old reader modules are still used in the eventbrowser, but I'm happy to leave the fix for that to a future PR
Maybe @cozzyd also wants to comment. AFAIK the uproot backend is fully functional. Yes mattak is not yet pip installed but I think it just have to be cloned and set in the Pythonpath for the uproot backend (correct @cozzyd). In that case it could be added into the install script (@sjoerd-bouma would you volunteer for that, I think you will be x10 faster than me). |
The uproot version does not need to be compiled, and should be pip-installable once RNO-G/mattak#18 has been merged. I've moved pandas over to be an optional dependency for now. |
…ter the default arugments for the MattakReader changed
|
@sjoerd-bouma sorry, I guess I need a new approval |
|
Thanks Felix, this is awesome 🥇 |
The data reader is based on code from @KelliMichaels. I added a few features such as a (optional) preliminary baseline correction, voltage calibration, and on the fly run selection. It is also possible to filter for specific trigger types (or any other information contained in the eventInfo class).
The noise reader is quite simple at the moment. The main issue I have with it right now is that is caches all noise event in memory at the beginning with can become impractical quite soon.