Key error for multiprocessing if not making raw_records

[JoranAngevaare]

There is an annoying key error in multiprocessing when raw_records, is already stored. See the example below:

>>>st_mp.make(run_id, 'event_info_double', max_workers = 10)
Removing old incomplete data in ./strax_data_mp/009104-veto_regions-pamvninlop
Removing old incomplete data in ./strax_data_mp/009104-records-pamvninlop
Removing old incomplete data in ./strax_data_mp/009104-pulse_counts-pamvninlop
Removing old incomplete data in ./strax_data_mp/009104-peaklets-ioudwhzz4n
Removing old incomplete data in ./strax_data_mp/009104-lone_hits-ioudwhzz4n
Removing old incomplete data in ./strax_data_mp/009104-peaklet_classification-nnlrqza4wr
Removing old incomplete data in ./strax_data_mp/009104-merged_s2s-4os2w32txl
Removing old incomplete data in ./strax_data_mp/009104-peak_basics-3k6xvmmrww
Removing old incomplete data in ./strax_data_mp/009104-peak_proximity-npglcajfmc
Removing old incomplete data in ./strax_data_mp/009104-events-zf5ybclhvk
Removing old incomplete data in ./strax_data_mp/009104-peak_positions-5ihezjokah
Removing old incomplete data in ./strax_data_mp/009104-event_basics-ahgsk5qm5w
Removing old incomplete data in ./strax_data_mp/009104-event_positions-dmugiittv4
Removing old incomplete data in ./strax_data_mp/009104-corrected_areas-nkrgct37wh
Removing old incomplete data in ./strax_data_mp/009104-energy_estimates-pztdnyajof
Removing old incomplete data in ./strax_data_mp/009104-event_info-j6jc4vuyjl
Removing old incomplete data in ./strax_data_mp/009104-distinct_channels-wj55i6dlbb

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-19-d0db1509c592> in <module>
----> 1 st_mp.make(run_id, 'event_info_double', max_workers = 10)

/mnt/d/Google_Drive/PhD-master/ubuntu-storage/ubuntu-windows/software/strax_official/strax/context.py in make(self, run_id, targets, save, max_workers, progress_bar, _skip_if_built, **kwargs)
   1097             return
   1098 
-> 1099         for _ in self.get_iter(run_ids[0], targets,
   1100                                progress_bar=progress_bar,
   1101                                save=save, max_workers=max_workers, **kwargs):

/mnt/d/Google_Drive/PhD-master/ubuntu-storage/ubuntu-windows/software/strax_official/strax/context.py in get_iter(self, run_id, targets, save, max_workers, time_range, seconds_range, time_within, time_selection, selection_str, keep_columns, _chunk_number, progress_bar, **kwargs)
    918 
    919         seen_a_chunk = False
--> 920         generator = strax.ThreadedMailboxProcessor(
    921                 components,
    922                 max_workers=max_workers,

/mnt/d/Google_Drive/PhD-master/ubuntu-storage/ubuntu-windows/software/strax_official/strax/processor.py in __init__(self, components, allow_rechunk, allow_shm, allow_multiprocess, allow_lazy, max_workers, max_messages, timeout)
     91                     int(np.argmin([len(p.depends_on)
     92                                    for p in mp_plugins.values()]))]
---> 93                 components = strax.ParallelSourcePlugin.inline_plugins(
     94                     components, start_from, log=self.log)
     95                 self.components = components

/mnt/d/Google_Drive/PhD-master/ubuntu-storage/ubuntu-windows/software/strax_official/strax/plugin.py in inline_plugins(cls, components, start_from, log)
    887         p.start_from = start_from
    888         if p.multi_output:
--> 889             p.dtype = {d: p.sub_plugins[d].dtype_for(d)
    890                        for d in outputs_to_send}
    891         else:

/mnt/d/Google_Drive/PhD-master/ubuntu-storage/ubuntu-windows/software/strax_official/strax/plugin.py in <dictcomp>(.0)
    887         p.start_from = start_from
    888         if p.multi_output:
--> 889             p.dtype = {d: p.sub_plugins[d].dtype_for(d)
    890                        for d in outputs_to_send}
    891         else:

KeyError: 'lone_hits'

Of course you only hit this if you are multiprocessing:

>>>st_mp.context_config

{'check_available': ('raw_records', 'peak_basics'),
 'store_run_fields': ('name', 'number', 'start', 'end', 'livetime', 'mode'),
 'storage_converter': False,
 'fuzzy_for': (),
 'fuzzy_for_options': (),
 'allow_incomplete': False,
 'allow_rechunk': False,
 'allow_multiprocess': True,
 'allow_shm': True,
 'allow_lazy': False,
 'forbid_creation_of': ('raw_records',
  'raw_records_he',
  'raw_records_aqmon',
  'raw_records_nv',
  'raw_records_aqmon_nv',
  'raw_records_aux_mv',
  'raw_records_mv'),
 'max_messages': 10,
 'timeout': 180,
 'use_per_run_defaults': False,
 'free_options': (),
 'apply_data_function': (<function straxen.common.remap_old(data, targets, works_on_target='')>,)}

I will shortly push a fix.

[JoranAngevaare]

fixed in #376