RadarCollection - Merged radar instances

[nguy]

Following on a previous post, we've been talking about merging multiple radar instances. There was no clear way merge the metadata specifically as I recall. Just to bring this up again and to point out one type of solution that has been started. Take a look at the way @deeplycloudy cloudy begins to handle this.

Issues that remain are the metadata and the fact that this uses Pandas. I have spoken with @jjhelmus previously and we though that adding another dependency would not be good. But maybe exploring the Pandas or xray could be a way to implement something in Py-ART?

This could also be very powerful for ARTView. I've been considering ways to cache radar instances for improved performance and potentially make animations.

I'd love to hear some thoughts.
@gamaanderson @scollis @kirknorth @tjlang

[scollis]

Time of pandas, I know there is a lot of people talking of moving it into the scipy core

-sent from a mobile device-

On Dec 10, 2015, at 12:13 PM, Nick notifications@github.com wrote:

Following on a previous post, we've been talking about merging multiple radar instances. There was no clear way merge the metadata specifically as I recall. Just to bring this up again and to point out one type of solution that has been started. Take a look at the way @deeplycloudy cloudy begins to handle this.

Issues that remain are the metadata and the fact that this uses Pandas. I have spoken with @jjhelmus previously and we though that adding another dependency would not be good. But maybe exploring the Pandas or xray could be a way to implement something in Py-ART?

This could also be very powerful for ARTView. I've been considering ways to cache radar instances for improved performance and potentially make animations.

I'd love to hear some thoughts.
@gamaanderson @scollis @kirknorth @tjlang

—
Reply to this email directly or view it on GitHub.

[nguy]

By the way for the packages I've been working on, I convert the time to Epoch for a package standard. Whether this is best or not, shrug, but it's what I did in the pyart.core.radar.join_radar function.

[jjhelmus]

From a quick look at use of pandas in the RadarCollection class from brawl4d I believe the same results could be achieved using a NumPy recarray. I'm not convinced that pandas or xray are really needed in this case.

A good discussion that I would like to have before designing a RadarCollect (and likely a GridCollection class) are what the the use cases and needs of such an object. With those requirement fleshed out it would be easier to discuss how best these can be implemented.

[scollis]

Well one would be a quick way to go from a radar collection to a grid
collection

Hell, some time we can work out how to use Jupyter cluster or
multiprocessing to make it easier for users with clusters to map reduce
lots of data..

also imagine being able to go from a radar collection to a QVP

Jonathan J. Helmus wrote:

From a quick look at use of pandas in the RadarCollection class from
brawl4d I believe the same results could be achieved using a NumPy
recarray
http://docs.scipy.org/doc/numpy-1.10.1/reference/generated/numpy.recarray.html.

I'm not convinced that pandas or xray are really needed in this case.

A good discussion that I would like to have before designing a
RadarCollect (and likely a GridCollection class) are what the the
use cases and needs of such an object. With those requirement fleshed
out it would be easier to discuss how best these can be implemented.

—
Reply to this email directly or view it on GitHub
#435 (comment).

[gamaanderson]

Just stopping by to remember of issue #252.

@nguy creating a plugin to automatic rotate a list of cached radars (simple radar, no Vradar) is not difficult, it would be good to test it.

[nguy]

Other uses are merging sweep files into a single collection (either ground-based or airborne), which also lead to a natural device for calculations: dual-Doppler, VAD, frequency by altitude analyses, tracking such as TITAN-esque, etc.

Good points @gamaanderson

[mgrover1]

Coming back to this, especially in context of the QVP functionality... should we have adopt an xarray dataset as this collection of radar instances?

I am thinking the QVP function could return something similar to the column extraction code. @scollis @zssherman @jrobrien91 thoughts here?

[mgrover1]

Merging with #670