NuRadioMC 3.0 #680

cg-laser · 2024-05-21T13:45:48Z

This PR is a complete refactor of the NuRadioMC core (simulation.py), enabling long-awaited features such as:

separate simulation of trigger channels
non-consecutive channel ids
enable resimulation from nur files
adding additional antennas on-the-fly to existing simulations
Exposing functions to calculate voltage traces from showers for stand-alone use

The key was to change the internal looping structure from

to

Changes will be discussed in upcoming InIceMC calls. More information will be provided to the PR later.

…olation model

…ay_tracing class

In order to use the trace multipolication for the birefringence calculations the branches were merched.

Merge branch 'develop' of https://github.com/nu-radio/NuRadioMC into birefringence

fschlueter · 2024-05-23T11:14:22Z

@cg-laser it looks like the diff includes the changes of the birefringent2 branch which was merged. Can you try to remove it from the PR? Maybe by either merging or rebasing the current development branch in this one?

NuRadioReco/modules/efieldToVoltageConverter.py

fschlueter · 2024-05-23T13:06:54Z

NuRadioReco/modules/efieldToVoltageConverter.py

-        for iCh in det.get_channel_ids(sim_station_id):
+        if channel_ids is None:
+            channel_ids = det.get_channel_ids(sim_station_id)
+        for iCh in channel_ids:


you could use the same variable as in the previous loop (channel_id)

NuRadioReco/modules/efieldToVoltageConverterPerEfield.py

fschlueter · 2024-05-23T13:09:00Z

NuRadioReco/modules/efieldToVoltageConverterPerEfield.py

+
+        Parameters
+        ----------
+        evt : Event


Sjoerd can comment better, but we could use references to the class object here which is reflected in the deployed documentation

sjoerd-bouma · 2024-05-23T13:14:36Z

One more question with regards to the new looping structure - how would you treat e.g. coherent noise sources (like the galaxy)? This is currently still WIP (see PR #591), but eventually we will probably want to have galactic noise that is coherent between different antennas (I'm not sure about deep detectors, but certainly for e.g. LOFAR this will be useful for calibration). How would that fit into the new looping structure?

fschlueter · 2024-05-28T07:41:28Z

NuRadioMC/simulation/simulation.py

+# initialize a few NuRadioReco modules
+# TODO: Is this the best way to do it? Better to initialize them on demand
+channelAddCableDelay = NuRadioReco.modules.channelAddCableDelay.channelAddCableDelay()
+electricFieldResampler = NuRadioReco.modules.electricFieldResampler.electricFieldResampler()
+efieldToVoltageConverterPerEfield = NuRadioReco.modules.efieldToVoltageConverterPerEfield.efieldToVoltageConverterPerEfield()
+efieldToVoltageConverter = NuRadioReco.modules.efieldToVoltageConverter.efieldToVoltageConverter()
+channelSignalReconstructor = NuRadioReco.modules.channelSignalReconstructor.channelSignalReconstructor()
+channelResampler = NuRadioReco.modules.channelResampler.channelResampler()
+channelGenericNoiseAdder = NuRadioReco.modules.channelGenericNoiseAdder.channelGenericNoiseAdder()
+channelResampler = NuRadioReco.modules.channelResampler.channelResampler()
+eventWriter = NuRadioReco.modules.io.eventWriter.eventWriter()
+triggerTimeAdjuster = NuRadioReco.modules.triggerTimeAdjuster.triggerTimeAdjuster()


Personally I am not a fan of calling the object the same as the module/class. If we would change that we could also simplify the module imports.

what is your suggestion?

same names but in snake_case? channelAddCableDelay -> channel_add_cable_delay or something similar like channel_cable_delay_adder

NuRadioMC/simulation/simulation.py

fschlueter · 2024-05-28T07:45:36Z

NuRadioMC/simulation/simulation.py

+    detector_simulation_filter_amp: function (optional)
+        a function that applies the filter and amplifier response to the electric field
+        the arguments to the function are (event, station, detector)
+        if not provided, the function `detector_simulation_part1` needs to be provided.


here and in the following it should be _part2 and not _part1. Also these names are a bit confusing. Can we find better names?

fschlueter · 2024-05-28T07:46:26Z

NuRadioMC/simulation/simulation.py

+        trace = np.zeros(n_samples)
+        trace[n_samples // 2] = 100 * units.V  # set a signal that should satisfy any trigger and speedup cuts
+        trace[n_samples // 2 + 1] = -100 * units.V
+        electric_field.set_trace(np.array([np.zeros(n_samples), trace, trace]), 1. / dt)
+        electric_field.set_trace_start_time(0)
+        electric_field[efp.azimuth] = 0
+        electric_field[efp.zenith] = 100 * units.deg
+        electric_field[efp.ray_path_type] = 0
+        sim_station.add_electric_field(electric_field)


Why is that necessary?

fschlueter · 2024-05-28T07:50:19Z

One general comment / question. I appreciated that Christoph's solution spread the code over several files. In my opinion that increases readability (of course there can be a to much files), instead of having one file with +1500 lines.

fschlueter · 2024-05-28T09:52:03Z

NuRadioReco/detector/antennapattern.py

                        os.path.basename(path)))
                    download_file = True
            else:
-                logger.warning("no hash sum of {} available, skipping up-to-date check".format(os.path.basename(path)))
+                logger.status("no hash sum of {} available, skipping up-to-date check".format(os.path.basename(path)))


I think this is actually worth a warning?

fschlueter · 2024-05-28T09:53:24Z

NuRadioReco/framework/base_station.py

+        efield_ids = []
+        for efield in self._electric_fields:
+            efield_ids.append(efield.get_unique_identifier())
+        efield_ids.sort()


I would not necessarily expect this sort here. Maybe add it to the information to the doc string

fschlueter · 2024-05-28T09:55:10Z

NuRadioReco/framework/base_station.py

@@ -322,3 +332,19 @@ def deserialize(self, data_pkl):
                self.set_station_time(data['_station_time'])

        self._particle_type = data['_particle_type']
+
+
+    def __add__(self, x):


doc string missing. Is it intuitive to implement such a operator? Maybe we should implement a merge or combine function

fschlueter · 2024-05-28T09:58:11Z

NuRadioReco/framework/event.py

        """
        Get an iterator over all simulated showers in the event
        """
-        for shower in self.__sim_showers.values():
-            yield shower
+        return self.__sim_showers.values()
+        # for shower in self.__sim_showers.values():
+        #     yield shower


change the doc-string. Typically this kind of functions returns an iterator right? Why changing that for this one?

fschlueter · 2024-05-28T10:06:22Z

NuRadioMC/simulation/simulation.py

        if det is None:
-            logger.status("Detectorfile {}".format(os.path.abspath(self._detectorfile)))
-            kwargs = dict(json_filename=self._detectorfile, default_station=default_detector_station,
+            logger.status("Detectorfile {}".format(os.path.abspath(detectorfile)))
+            kwargs = dict(json_filename=detectorfile, default_station=default_detector_station,
                              default_channel=default_detector_channel, antenna_by_depth=False)
            kwargs.update(det_kwargs)
            self._det = detector.Detector(**kwargs)
        else:
            self._det = det


Can we break backwards compatibility? If so I am opting for removing the initialisation of the detector object from this class. The object should be passed to the class and initialized in the top level script. This also forces the user to think about it a bit more.

We could also finally remove the use of the deprecated default_channel/station

cg-laser · 2024-06-06T13:00:13Z

One more question with regards to the new looping structure - how would you treat e.g. coherent noise sources (like the galaxy)? This is currently still WIP (see PR #591), but eventually we will probably want to have galactic noise that is coherent between different antennas (I'm not sure about deep detectors, but certainly for e.g. LOFAR this will be useful for calibration). How would that fit into the new looping structure?

Good question. This is important to consider. I think I would implement an internal buffer of the genericNoiseAdder. As long as the event_group_id is the same, the same noise is being used and just delayed to accommodate the difference in propagation times to the antennas. (Although I'm not sure how you would do this in practice as the emission comes from all directions but you must have figured this out. )

nilsphysics and others added 30 commits October 2, 2021 14:03

added birefringence IceModel to medium.py

6d10535

changed example IceModel_new file

12f52f0

changed the read in of the interpolation files, added smoother interp…

53aabc0

…olation model

added birefringence time delay and polarization calculations to the r…

02707ef

…ay_tracing class

Merge branch 'base_trace_multiplocation' into birefringence

bebb959

In order to use the trace multipolication for the birefringence calculations the branches were merched.

added pulse propagation, changed obj function (independent of omega

feb3c2b

pulse propagation faster and with adjustable pulse shape

e2b406d

added numpy file of modelB

4aa3890

streamlined method to read in new ice models

c69bb37

remove old files

f197574

adding more ice models, updating the ray tracer

7050136

updating the ray tracer and medium

38e1e42

analytical root finder method, fixed polarization bug

25da11d

analytical root finder method, fixed polarization bug

85ecdd6

bugfix

56a5383

Resolving Merge conflict in analytical ray tracer

61eb7d1

Merge branch 'develop' of https://github.com/nu-radio/NuRadioMC into birefringence

reworked the structure of the ice model

4626f89

debuged the special polarization cases

6ae7877

included birefringence in apply_propagation_effects

557e798

updated IceModels and documentation

a970a57

added a birefringence unit check to run_signal_test.sh

5ac4ab4

preparing for full simulation, bug fixes

cc5d8a0

fixing unit check

58ff329

fixing documentation error

67fe315

introduce birefringence to RadioPropa (not fully working yet)

fc0fa6f

working version of birefringence in RadioPropa

0a47dac

speed and accuracy improvements

ff38469

fixing birefringence test (1)

0854d2a

fixing birefringence test (2)

c9dd5eb

fixing birefringence test (3)

7cef814

bugfix

4f59f4a

cg-laser requested review from fschlueter, christophwelling and sjoerd-bouma May 21, 2024 13:45

cg-laser changed the base branch from develop to birefringence2 May 23, 2024 12:31

cg-laser changed the base branch from birefringence2 to develop May 23, 2024 12:31

cg-laser changed the base branch from develop to birefringence2 May 23, 2024 12:32

cg-laser changed the base branch from birefringence2 to develop May 23, 2024 13:04

fschlueter reviewed May 23, 2024

View reviewed changes

NuRadioReco/modules/efieldToVoltageConverter.py Show resolved Hide resolved

fschlueter reviewed May 23, 2024

View reviewed changes

NuRadioReco/modules/efieldToVoltageConverterPerEfield.py Show resolved Hide resolved

fschlueter reviewed May 23, 2024

View reviewed changes

cg-laser force-pushed the refactor branch from 1441cb4 to 4f59f4a Compare May 24, 2024 14:08

cg-laser added 4 commits May 24, 2024 14:19

Merge branch 'develop' into refactor

a622ec3

fix merge problems manually

2fa0041

add convenient function to retun all parameters

203db70

improve plotting

841c890

fschlueter reviewed May 28, 2024

View reviewed changes

NuRadioMC/simulation/simulation.py Show resolved Hide resolved

fschlueter reviewed May 28, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NuRadioMC 3.0 #680

NuRadioMC 3.0 #680

cg-laser commented May 21, 2024

fschlueter commented May 23, 2024

fschlueter May 23, 2024

fschlueter May 23, 2024

sjoerd-bouma commented May 23, 2024

fschlueter May 28, 2024

cg-laser Jun 6, 2024

fschlueter Jun 6, 2024

fschlueter May 28, 2024

fschlueter May 28, 2024

fschlueter commented May 28, 2024

fschlueter May 28, 2024

fschlueter May 28, 2024

fschlueter May 28, 2024

fschlueter May 28, 2024

fschlueter May 28, 2024

fschlueter May 28, 2024

cg-laser commented Jun 6, 2024

NuRadioMC 3.0 #680

Are you sure you want to change the base?

NuRadioMC 3.0 #680

Conversation

cg-laser commented May 21, 2024

fschlueter commented May 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sjoerd-bouma commented May 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fschlueter commented May 28, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cg-laser commented Jun 6, 2024