livdb_magic produces extra columns and is missing req'd columns

[ltauxe]

produces a very out of date data model 3.0 - it has many un-necessary columns (like sample, site) but is missing the required fields of treat_ac_field and sequence. It does add them but somehow measurements_methods3 removes them and i can't see where.

[moonshoes87]

measurements_methods3 is not called in livdb.

I tested this on the LIVDB_magic/TH_IZZI+ data.

The problem was that MeasRec['treat_ac_field'] wasn't initialized across all records. In non-AF studies, treat_ac_field is only set if there are no prior treatments AND peak field is not 999, -99 or 0. In the TH_IZZI+ data, the first peak field is 0. So the first MeasRec doesn't have the treat_ac_field column. Because magic_write assumes that all records will have the same columns as the first record, treat_ac_field is not written for any of the records.

This is the logic in livdb that sets treat_ac_field:

                        no_treatments_yet = True
                        for t in experiment_treatments:
                            t = float(t.strip())
                            if t > 50:
                                no_treatments_yet=False
                        #print(' no treatments yet?', no_treatments_yet)                                                       
                        if no_treatments_yet:
                            #print( 'af demag remag peak field', meas_line['AF Demag/Remag Peak Field'])                       
                            if meas_line['AF Demag/Remag Peak Field'] != "" and \
                               float(meas_line['AF Demag/Remag Peak Field']) != 999 and \
                               float(meas_line['AF Demag/Remag Peak Field']) != -99 and \
                               float(meas_line['AF Demag/Remag Peak Field']) != 0:
                                lab_treatment = "LT-AF-Z"
                                MagRec[ac_field_col] = '%8.3e' % (
                                    float(meas_line['AF Demag/Remag Peak Field'])*1e-3)  # peak field in tesla 

I can solve this problem by initializing MagRec[ac_field_col] = "" in all cases. @ltauxe, let me know if there's something I'm not understanding about how this should work.

There are no AF examples in the example data, but this is the only other place where treat_ac_field can be set.

                        elif "AF-" in Experiment_Type:
			MagRec[ac_field_col] = '%8.3e' % (
                                float(meas_line['TH Demag/Remag Peak Temperature'])*1e-3)  # peak field in tesla               
                            treatment = meas_line['AF Demag/Remag Peak Field']

[ltauxe]

Yes I think that will work - I can test it on other examples

…

On Mon, Aug 20, 2018 at 4:55 PM Lori J ***@***.***> wrote: measurements_methods3 is not called in livdb. I tested this on the LIVDB_magic/TH_IZZI+ data. The problem was that MeasRec['treat_ac_field'] wasn't initialized across all records. In non-AF studies, treat_ac_field is only set if there are no prior treatments AND peak field is not 999, -99 or 0. In the TH_IZZI+ data, the first peak field is 0. So the first MeasRec doesn't have the treat_ac_field column. Because magic_write assumes that all records will have the same columns as the first record, treat_ac_field is not written for any of the records. This is the logic in livdb that sets treat_ac_field: no_treatments_yet = True for t in experiment_treatments: t = float(t.strip()) if t > 50: no_treatments_yet=False #print(' no treatments yet?', no_treatments_yet) if no_treatments_yet: #print( 'af demag remag peak field', meas_line['AF Demag/Remag Peak Field']) if meas_line['AF Demag/Remag Peak Field'] != "" and \ float(meas_line['AF Demag/Remag Peak Field']) != 999 and \ float(meas_line['AF Demag/Remag Peak Field']) != -99 and \ float(meas_line['AF Demag/Remag Peak Field']) != 0: lab_treatment = "LT-AF-Z" MagRec[ac_field_col] = '%8.3e' % ( float(meas_line['AF Demag/Remag Peak Field'])*1e-3) # peak field in tesla I can solve this problem by initializing MagRec[ac_field_col] = "" in all cases. @ltauxe <https://github.com/ltauxe>, let me know if there's something I'm not understanding about how this should work. There are no AF examples in the example data, but this is the only other place where treat_ac_field can be set. elif "AF-" in Experiment_Type: MagRec[ac_field_col] = '%8.3e' % ( float(meas_line['TH Demag/Remag Peak Temperature'])*1e-3) # peak field in tesla treatment = meas_line['AF Demag/Remag Peak Field'] — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#418 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKrDaCGquxHrfhEkCnW7yizjyllRwJyks5uS0x3gaJpZM4V-rFJ> .

[moonshoes87]

@ltauxe, I also fixed livdb so it will generate samples/sites/locations tables if possible. Let me know if you have more problems here.

[moonshoes87]

New error:

-W- No intensity data found for specimens
Traceback (most recent call last):
  File "/Users/nebula/Python/PmagPy/programs/thellier_gui.py", line 8182, in <module>
main()
  File "/Users/nebula/Python/PmagPy/programs/thellier_gui.py", line 8175, in main
app.frame = Arai_GUI(WD)
  File "/Users/nebula/Python/PmagPy/programs/thellier_gui.py", line 405, in __init__
self.InitUI()
  File "/Users/nebula/Python/PmagPy/programs/thellier_gui.py", line 426, in InitUI
self.Add_text(self.s)  # write measurement data to text box
  File "/Users/nebula/Python/PmagPy/programs/thellier_gui.py", line 1107, in Add_text
for d in self.Data[s]['datablock']]
  File "/Users/nebula/Python/PmagPy/programs/thellier_gui.py", line 1107, in <listcomp>
for d in self.Data[s]['datablock']]
AttributeError: 'int' object has no attribute 'split'
/Users/nebula/anaconda3/bin/pythonw: line 3: 89248 Segmentation fault: 11      /Users/nebula/anaconda3/python.app/Contents/MacOS/python "$@"

[moonshoes87]

@ltauxe, this is kinda rough. I'm using the data_files/Measurement_Import/LIVDB_magic/MW_C+ data.

The problem is that Thellier GUI is trying to index on the list of 'treatment_mw_power' values:

[0, 10, 10, 20, 18, 25, 25, 25, 6, 28, 28, 30, 30, 30, 25, 33, 33, 35, 35, 40, 40, 40, 30]

But, the values in the drop down boxes for tmin/tmax are:

['0', '1', '2', '3', '4', '5', '6', '7', '8']

which is generated from Step number in the description column:

Step Type-NRM:Step Number-0

which really isn't the mw power at all. Also, this is very specific to a particular format... What do you think?

[moonshoes87]

@ltauxe, I put in a possible solution for this. Give it a try and see if it seems like the expected behavior to you. It's a little odd, how this was set up....

[ltauxe]

will do

…

On Aug 22, 2018, at 12:46 PM, Lori J ***@***.***> wrote: @ltauxe <https://github.com/ltauxe>, I put in a possible solution for this. Give it a try and see if it seems like the expected behavior to you. It's a little odd, how this was set up.... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#418 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAKrDSlsxRvunSoHaFlOn3rvMy86KIUCks5uTbUjgaJpZM4V-rFJ>.

[ltauxe]

this seems to work ok now.