You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Could you refactor raw_data.py to make M5 instances to return not only numpy array with chunk of dynamical spectra, but also metadata dictionary with the following keys:n_nu, n_t, nu_0, d_nu, d_t, t_0, exp_code, antenna, freq, band ,pol? I finally merged metadata with dynamical spectra array in one class. After creating DynSpectra instances with array & metadata we don't need anything else for subsequent processing.
It could be cool if method that does this will return generator - not to store all arrays in memory but creating them from Mark5 only when they are needed in small chunks (like 5 minutes or even less).
Another option - to put data in HDF5 format that can save metadata too. See utils.save_hdf5 as example.
The text was updated successfully, but these errors were encountered:
Finally i polished dyn_spectra.DynSpectra API to level when it is the only thing we need to go further (manage pipeline etc.). Now, to proceed, i need M5 (or whatever) instances to create instances of DynSpectra class (one instance for each chunk of Mark5 file). This part is for you, @akutkin ;) I think it would be easy cause DynSpectra creating needs only dyn. spectra array + metadata with fixed keys.
Could you refactor
raw_data.py
to makeM5
instances to return not onlynumpy
array with chunk of dynamical spectra, but also metadata dictionary with the following keys:n_nu
,n_t
,nu_0
,d_nu
,d_t
,t_0
,exp_code
,antenna
,freq
,band
,pol
? I finally merged metadata with dynamical spectra array in one class. After creatingDynSpectra
instances with array & metadata we don't need anything else for subsequent processing.It could be cool if method that does this will return generator - not to store all arrays in memory but creating them from Mark5 only when they are needed in small chunks (like 5 minutes or even less).
Another option - to put data in
HDF5
format that can save metadata too. Seeutils.save_hdf5
as example.The text was updated successfully, but these errors were encountered: