-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement dictionaries for power and temperature sweep data #58
Comments
I'm not really sure what you mean by using python dictionaries to organize sets of data. Are you saying dynamically during fitting it will keep each data set in a dictionary? If so I don't believe that's necessary because all we really need are the Qi and Qc values from each iteration. |
@crmcrae I'm a big fan of using data structures like this when we can. It will simplify the code and make it more readable. To add to that, there's a special thing called a dataclass that is like a dictionary but even more useable that we can use here. For example we can create a DataSet class like: @dataclass
class DataSet:
freqs: Array
amplitudes: Array
phases: Array then when we want to put the information in the dataset we do new_data = DataSet(freqs = [....], ampliudes = [....], phases = [....]) and we can now access all the attributes really simply, for example when we plot amplitude vs frequency we can write:
|
In the case where your data is multiple vectors that share a common
coordinate vector (i.e. frequency is the coordinate vector, and phase,
magnitude, I, Q, etc are the data vectors) then either a pandas dataframe
or an xarray dataset seem like more reasonable options (both provide built
in hooks to plotting, broadcasting for element-by-element operations,
slicing, etc etc). Xarray is especially nice because you can have multiple
coordinates (i.e. temperature, power, magnetic field, whatever) and you can
slice across them more or less in any way you like. You can also then store
those datasets in a compact binary representation (netcdf4 or hdf5 format).
You also get dot-completion access for both (data.freqs, data.I,
data.phase, etc).
…-FC
On Wed, Mar 11, 2020 at 2:57 PM Josh Mutus ***@***.***> wrote:
@crmcrae <https://github.com/crmcrae> I'm a big fan of using data
structures like this when we can. It will simplify the code and make it
more readable. To add to that, there's a special thing called a dataclass
<https://docs.python.org/3/library/dataclasses.html> that is like a
dictionary but even more useable that we can use here.
For example we can create a DataSet class like:
@dataclassclass DataSet:
freqs: Array
amplitudes: Array
phases: Array
then when we want to put the information in the dataset we do
new_data = DataSet(freqs = [....], ampliudes = [....], phases = [....])
and we can now access all the attributes really simply, for example when
we plot amplitude vs frequency we can write:
plt.plot(new_data.freqs, new_data.amplitudes)
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#58 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACTQRRH7MGFNO32G3HQNIBDRHACLXANCNFSM4KRX3R2Q>
.
|
That sounds great, thanks Faustin! @mullinska do you have a preference between pandas and xarray? |
I have used pandas before, but I'm sure that xarray isn't that hard to figure out so I really don't have a strong preference either way. As it stands right now the data is just being stored in numpy arrays inside of a data keeping class. |
In my resonator analysis package (scraps) I generally used pandas for
storing fit parameters as functions of temperature and input power. Once
xarray became minimally stable I also started rewriting the entire thing to
use xarray as the base data store with all the class methods as properties
on top of that. I never really finished integrating it accross the whole
package, but the resonator class code is here:
https://github.com/FaustinCarter/scraps/blob/xarray/scraps/resonator.py and
you can feel free to borrow as much as you like.
In general xarray is an N-dimensional generalisation of a pandas series (in
much the same way that a DataFrame is a 2D series). I'm not sure what the
right choice is, but xarray has some nice features for storing metadata (in
pandas that's not really supported), concatenating data along any axis,
dumping out to a usable binary form, etc.
I think pandas probably has a simpler interface, but you can always dump
any two dimensions from an xarray out to a pandas dataframe with minimal
effort (and vice versa).
The downside is that they are developing very actively and sometimes
features get deprecated, or breaking changes are introduced.
I went with xarray because I wanted to support an arbitrary number of
"knobs" like magnetic field, temperature, VNA power, etc. and I really
wanted to store all my metadata (IF bandwidth, VNA model, resonator
material, etc) in the same file as my data (in a compact, standardized
binary format).
…-FC
On Tue, Mar 17, 2020 at 1:41 PM mullinska ***@***.***> wrote:
I have used pandas before, but I'm sure that xarray isn't that hard to
figure out so I really don't have a strong preference either way. As it
stands right now the data is just being stored in numpy arrays inside of a
data keeping class.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#58 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ACTQRRHL6TYWUYCZZQHG6TDRH7OA3ANCNFSM4KRX3R2Q>
.
|
Hi @FaustinCarter xarray looks like a really interesting option. |
@FaustinCarter would you be interested in porting over your data structure into this repo? It might be a nice contained project. |
Hey @joshmutus, I'm ok with this idea, but it's not clear to me exactly how you want it implemented. Is the idea that this would replace the Resonator class currently located here by using the guts of scraps (xarray based storage with propery-based methods) and adding in the necessary hooks for cryores to use it: Or did you want it implemented in some other way? Another option is that I would be willing to put some work into refactoring and extending the scraps code so that you could just import the scraps class directly (i.e. making scraps a dependency). This might make future updating easier as there would then be a single source for the code. Let me know what you think and I can try and figure out how to go about it. It might take a bit of time since I'd mostly have to work on it in my spare time. |
Hi @FaustinCarter I saw this post so I thought I would clear up that in the last major push that @joshmutus made, the data is no longer stored in the Resonator class. The current place the data is being stored is: Where there is an option to read in the data from a .csv file or to feed the data directly into the function. The old place the data was being stored (the Resonator class) isn't really being used for anything at the moment, so if the data struct were to be implemented it would need to take over this function. |
I propose we use Python dictionaries to organize sets of data for various powers and temperatures, which our TLS fitting and power/temp sweep plotting tools would then read in. What do you think @mullinska ?
The text was updated successfully, but these errors were encountered: