Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor M5 to follow API of subsequent processing #10

Closed
ipashchenko opened this issue May 25, 2016 · 2 comments
Closed

Refactor M5 to follow API of subsequent processing #10

ipashchenko opened this issue May 25, 2016 · 2 comments

Comments

@ipashchenko
Copy link
Collaborator

Could you refactor raw_data.py to make M5 instances to return not only numpy array with chunk of dynamical spectra, but also metadata dictionary with the following keys:n_nu, n_t, nu_0, d_nu, d_t, t_0, exp_code, antenna, freq, band ,pol? I finally merged metadata with dynamical spectra array in one class. After creating DynSpectra instances with array & metadata we don't need anything else for subsequent processing.

It could be cool if method that does this will return generator - not to store all arrays in memory but creating them from Mark5 only when they are needed in small chunks (like 5 minutes or even less).

Another option - to put data in HDF5 format that can save metadata too. See utils.save_hdf5 as example.

@ipashchenko
Copy link
Collaborator Author

Finally i polished dyn_spectra.DynSpectra API to level when it is the only thing we need to go further (manage pipeline etc.). Now, to proceed, i need M5 (or whatever) instances to create instances of DynSpectra class (one instance for each chunk of Mark5 file). This part is for you, @akutkin ;) I think it would be easy cause DynSpectra creating needs only dyn. spectra array + metadata with fixed keys.

@ipashchenko
Copy link
Collaborator Author

resovled with 2866a0f

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants