New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
doesn't load in STATIC energy / phi / theta / etc data #168
Comments
I'm having the same issue as @NeeshaRS. I'm trying to pull STATIC's energy, mass and eflux data. |
I was looking over this with Neesa yesterday, and the problem is the Energy bins (labeled as "energy" inside the file) have an attribute VAR_TYPE of "metadata", rather than "data" or "support_data". I assume that's actually a mistake within the STATIC files, because energy should definitely be considered "support_data". But alas, there is no way they're going to change the file format at this point. The cdf_to_tplot routine assumes that all things labeled as metadata are either options to input into tplot (ylog, spec, title, etc), or they are not needed. However in this case, we'd want to read in the metadata because its crucial to understanding the data in the file. As a quick fix, on the mpl and master branches of pytplot, I added a "get_metadata" option to the cdf_to_tplot routine so that it at least reads in the data so they can access the energy bins. I'm not sure if there is a much better way of handling it, but that's what I've done for now. |
Thanks Bryan! I'll go ahead and update the STATIC routine to use the new keyword automatically (and release a new version of the MPL pytplot so it all just works) |
Oops - didn't mean to close this, though it should be fixed now as of v1.3.12 |
Looks like the update broke something in pytplot... |
BUT the original problem we asked about does seem to have been fixed-- thank you! |
From the error, it looks like the eflux variable has too many dimensions. Is it a function of angle as well as energy? One thing to note: in IDL SPEDAS, we have ~135k lines of code in the 'maven' folder alone, so there are many, many support routines that haven't been ported over yet just for MAVEN. It's possible that one of these routines does some calculations with these eflux variables prior to loading (e.g., averaging over angle) - but I haven't checked yet. |
I looked into this some more; from the IDL crib sheet: https://github.com/spedas/bleeding_edge/blob/master/projects/maven/sta/mvn_sta_l2_crib.pro It looks like it loads the data from the CDF files with mvn_sta_l2_load, then produces tplot variables with mvn_sta_l2_tplot: https://github.com/spedas/bleeding_edge/blob/master/projects/maven/sta/mvn_sta_l2_tplot.pro So this file should have what you need to produce the tplot variables. The eflux is indeed multi-dimensional, so you'll definitely have to do some post-processing to create the variables you're interested in, e.g., from mvn_sta_l2_tplot.pro:
|
Yeah, in IDL I use those functions and work it that way. @juditbergfalk and I were just surprised because that function in python used to work and now it doesn't. I will dig into it a bit too (a lot of meetings today so I'll be in & out in my poking of things)... but in IDL when tplots are multidimensional, if they are 3D (like in your example) it plots a spectrogram. As far as I can tell, tplots are never more than 3D. |
Always happy to help! In IDL tplot, if the variable is multi-dimensional, it'll just use the first element of the last dimension, e.g., if eflux were [time, energy, angle], the plot would be an energy vs. time spectrogram for the first angle only. |
I agree with @NeeshaRS, the original problem is fixed. Thank you for that! |
Do you have a simple example that would allow me to reproduce the problem? |
Here is an example:
|
Thanks for the example! I just checked in a fix for the crash with As for the error with colors: I don't think matplotlib supports the color format you're using for the SWIA T pseudo-variable :
When I try one of these on a normal tplot variable, I get the error:
Using the color formats specified in: https://matplotlib.org/stable/tutorials/colors/colors.html should fix the crash you're seeing; e.g., using:
worked for me. One more note: Hope this helps! |
Thank you for your help always! |
What environment are you running in? If you're running it locally, so each call to pytplot.tplot will pause the script until you close the tplot window. Running inside a Jupyter notebook shouldn't have this issue. If you're running locally and want to create multiple plots without having to close the tplot window, I suggest setting the 'display' keyword to False and saving the figures using save_png (or save_svg/save_pdf/save_eps). I was able to run your script on Colab: https://colab.research.google.com/drive/15_8696zPkkBBB4FKCuFvP8Y_cL35Oc92?usp=sharing after changing the colors on the SWIA temp variable. Note: the 'slice' option to tplot also isn't implemented yet for the matplotlib version of pytplot. |
were you two able to sync up? @juditbergfalk had recommended doing a zoom with @supervised -- would that be possible? I feel like a 15min zoom would quickly get you both on the same page |
Thank you @NeeshaRS . I agree, a Zoom meeting would probably a good idea if you're available @supervised . I'm just a little bit confused because both the slice and the colors worked before the update. |
The backend to PyTplot (the entire part that renders figures) has been completely re-written from the ground up over the last 8 months; the version that comes with PySPEDAS now uses a completely different library internally (matplotlib vs. Qt previously). The slice keyword isn't there because it hasn't been implemented in the new version yet, and the color keyword doesn't accept the previous values because the way of specifying colors in matplotlib is different than Qt. You can find more information here: I'll look into adding the slice functionality back. I can setup a Zoom meeting if you need, would you be available Thursday morning at 11AM Mountain time? |
Oh okay, that makes sense now. Sorry for not understanding. I guess I should have updated it. I was really pleased to see the slice option and was even telling @NeeshaRS how cool of a tool it is to have so I'm looking forward to see that again but I understand if it'll be a lot of work! |
The latest release of the matplotlib version of pytplot (v2.0.9 and later) now supports the 'slice' keyword; you can upgrade with: pip install --upgrade pytplot-mpl-temp Hope this helps! |
I was trying to use pyspedas to pull in STATIC data but I noticed large discrepancies in the data pulled by PySPEDAS vs pulling data in IDL. Looking at the CDF file, it seems like pyspedas is ignoring the file's metadata and just pulling variables/support variables. Unfortunately, for STATIC, important parameters (like energy, phi, theta, etc) are given in metadata rather than variables/support variables.
Please enable reading in of the metadata parameters so STATIC data can be used to the full extent in python as well as in IDL.
Thank you!
The text was updated successfully, but these errors were encountered: