Expressions returning a nul result?

[NicoleRayner]

In my custom expressions I have some expressions that DUPLICATE pre-existing expressions but just rename them. This was necessary given the limitations that equation=column in SQUID2. I know it won't be necessary any more with SQUID3 but there is some strange behaviour here that I would like to understand. If you examine the CUSTOM expression MF corr. 4-corr 206Pb/238U it calls on the built in expression 4-corr 206/238

However in the peak, it returns all 0's for the age and uncertainty.
The built in expression that it calls on returns the correct values. Why?

[bowring]

The built-in expression is giving value for ref materials (check tabs in peek); add a "S" to the end of your expression: ... ["*4-corr 206*/238*S*"] to get the expression for unknowns

…

On Sat, Sep 1, 2018 at 5:10 AM NicoleRayner ***@***.***> wrote: In my custom expressions I have some expressions that DUPLICATE pre-existing expressions but just rename them. This was necessary given the limitations that equation=column in SQUID2. I know it won't be necessary any more with SQUID3 but there is some strange behaviour here that I would like to understand. If you examine the CUSTOM expression MF corr. 4-corr 206*Pb/238U it calls on the built in expression 4-corr 206*/238 [image: an expression that duplicates an existing expression] <https://user-images.githubusercontent.com/40035242/44931425-11f49d00-ad51-11e8-851a-fd667b68bd98.JPG> However in the peak, it returns all 0's for the age and uncertainty. The built in expression that it calls on returns the correct values. Why? [image: an expression that duplicates an existing expression part 2] <https://user-images.githubusercontent.com/40035242/44931504-5ed87380-ad51-11e8-88e9-6ae9719a906f.JPG>

[NicoleRayner]

Right. Of course.
What was adding confusion for me was that the corresponding expression related to MF-corr 206/238 AGE did not have a suffix indicating they were for unknown samples, and it worked just fine. This is because there are no comparable expressions for refmat and hence no need for a suffix. I now realize this in hindsight but it does make it difficult to troubleshoot expressions. I think this highlights the importance of being able to filter expressions for refmat (if this expression had shown up in a filtered list I would have known there was a problem). I also think it would be much better to add the suffix (_RM) to the refmats, rather than a _S suffix to unknowns. The refmats are kind of the exceptional case in expressions and so would be helpful to be flagged.
Thanks for the tip.

[bowring]

@sbodorkos and I have planned to formalize the naming of built-in expressions and this is a good trigger to do so, and a good idea of yours as well. Stay tuned!

[sbodorkos]

In terms the original issue raised by @NicoleRayner I think the expression validator could potentially be enhanced in order to help with that. It is clear in the graphic that the custom expression is for unknowns ONLY, and it seems clear that Squid3 "knew" that the named expression used was applicable to the reference material ONLY - it would be nice to receive a general warning about a conflict like that during validation, as well as having the expression labelled "unhealthy" in cases where there is no overlap at all between the destination of the custom expression (e.g. unknowns) and the names of the terms used in the expression (e.g. reference materials).

@bowring the list of built-in expression names you emailed me overnight looks incomplete, and reviewing Nicole's screenshots has reminded me that the sets of built-in expressions comprising any given Task are a function of which Permutation it is (my guess is you have sent me a list generated from a "Perm1" Task). So presumably I should generate a superset of names based on my collection of Perm1–Perm4 Tasks from Korea, and account for all the duplicates that way?

I also need a reminder of why some parameters occupy only one row in your list (e.g. RU-labelled "ppmU"), whereas similar-looking parameters occupy two rows (e.g. R-labelled "ppmTh" and U-labelled "ppmThS"). Is this because there is only a single expression used to evaluate ppmU (irrespective of R/U labelling), whereas ppmTh is evaluated using different expressions, depending on whether it is R- or U-labelled?
Presumably the list has one row for every distinct permutation of (expression name and expression arithmetic)?

[sbodorkos]

@bowring as a preliminary step, I have attached (ExpressionNames AcrossPerms_v1.xlsx) the reconciliation of the expression-names across all the Perms. I have examined all the grey gaps carefully and ensured that they do make sense. But we do have some issues that need addressing from a philosophical standpoint:

1. This list conflates expressions for Common Pb Models, RM Isotopic Models, RM Chemical Models, Special U-Th-Pb (which in isolation was the original definition of "built-in"), and Ludwig's "under the hood" arithmetic, which is a problem because not all of these expression-types are hierarchically equivalent. So not all of these expression can be sensibly-named (I think), although resolving the hierarchy issues might help with that.
   I understand that this range of expression-types are probably similar in terms of requiring consistent expression-names, but if we don't categorise them in an intelligible way, there will be widespread user confusion (to add to my own confusion, because I don't yet know what some of these expression-names represent!).

2. In the case of unknowns, I foresee the need for (at least) two different types of expression-name, to serve the dual purpose of (a) defining every single possible label-expression permutation uniquely ("unique name"), and (b) not burdening the user with unnecessary complexity that is irrelevant at best, and incorrect at worst ("usage name").
   The fundamental workflow (across every Squid, ever) is to reduce the data from the Isotopic Reference Material, in conjunction with one (or more) index isotopes for the common Pb correction (in fact, SQUID 2.50 permits data-reduction of RMs only, with all unknowns ignored). As we know, Ludwig's SQUIDs required the index isotope to be specified by the user prior to data-reduction; whereas Squid3 evaluates all the sensible candidate index-isotopes for a given Perm (i.e. 4corr and 7corr across all Perms, with the option of 8corr for Perm1 only).
   The thing is, it remains a fundamental requirement that the index isotope for common Pb correction of the RM be selected and finalised before much meaningful data-reduction of the unknowns can be done. This means that at the moment, we are calculating suites of contextually-equivalent but mutually-exclusive expressions for the unknowns, and their "unique names" contain detail that is unnecessary in any one given data-reduction, and which will also cause confusion for users.

An example of the latter (which uses the current expression-names) is the unknown-only expression currently named ["Total 206Pb/238US"]. This expression is always index-isotope-dependent, so we have a need to be able to calculate and store ["4-corr Total 206Pb/238US"] and ["7-corr Total 206Pb/238US"] for all Perms, as well as ["8-corr Total 206Pb/238US"] in the case of Perm1. Although all of these expression-names are uniquely and anatomically correct, they are not especially helpful for actual Squid3 users, both because they are counter-intuitive (the conventional theory is that ["Total 206Pb/238US"] is not dependent on the common Pb correction, and it's still not completely clear to me why Ludwig has formulated the arithmetic the way he has), and because they can never coexist: the index isotope selected for the RM always defines ONE value of ["Total 206Pb/238US"] for the unknowns.

Therefore, from a user perspective (i.e. in terms of the "usage" expression-names displayed in the left-hand lists in Squid3), it would be much more intuitive if there existed a single expression-name ["Total 206Pb/238US"], which was essentially an alias of the specific expression for the appropriate index isotope. A complication in this particular example is that all four Perms seem to already have a third (or fourth, in Perm1) expression named ["Total 206Pb/238US"], and it is not yet clear to me whether this is already the alias I have suggested, or whether it is a different expression for which the index-isotope was not specified at the time of calculation. I need to dig a bit deeper.

At the most fundamental level, this is why I would like to segregate built-in RM expressions from built-in Unknown expressions, at least in terms of "usage names", if not "unique names". For RMs, "unique names" will dominate, because we need to rigorously prefix and display everything, so the user can see all the index-isotope value-sets and make an informed choice about which one to use. Conversely, for the Unknowns, many of the rigorous prefixes required by the "unique names" can be dispensed with, in favour of more intuitive "usage names", because the relevant expression becomes uniquely defined through the preceding choice of RM index isotope.

[bowring]

One clarification relative to next-to-last-paragraph:

The four expressions in rows 98-99 and 151-154 ('Total ...") ARE in fact the aliases you describe - if you change the index isotope, these aliases are updated to refer to the corresponding explicitly-corrected expression when the user changes the selected isotope in the task manager. Note that both the explicitly-corrected expression and the alias are listed as built-in expressions since the latter is defined in terms of the former.

Other existing aliases that behave in the same way are: "ppmTh", "ppmThS", "232Th/238U", "232Th/238US", "204/206 (fr. 208)", "204 overcts/sec (fr. 208)", "8-corr Primary calib const. delta%". Each gets re-defined in terms of the underlying explicitly-corrected expression when the selected isotope is changed in the task manager.

[sbodorkos]

@bowring Excellent, this is what I was really hoping you were going to say! I will map all those aliases out, and that will help condense the long, complex list of "unique names" into a shorter, more intuitive list of "usage names".

[NicoleRayner]

@sbodorkos let me know if/how I can help with this process. One thought related to naming is that the flag indicating that the expression is applied to an unknown is "U" but in the list of custom expressions their names are suffixed by "S" for samples. Obviously "U" as a suffix has problems (conflict with uranium). I'm open to ideas on consistent naming conventions across the platform, I don't have a magic solution though. Perhaps flag remains as "U" but suffix is " _Unk"?

[sbodorkos]

@NicoleRayner the suggestion @bowring relayed to me (which I thought was a good one, and which I also think he attributed to you!) was to use disambiguation on the Standards (rather than the Unknowns), to exploit the fact that most people are writing their Tasks and expressions to work on Unknowns (rather than Standards). At present I am tinkering with using a prefix (rather than a suffix), and using the prefix "RM" where needed. Right now I am working on a first-pass for everything, which I will post here for comment.

[sbodorkos]

@NicoleRayner here's what I have done so far (attached ExpressionNames AcrossPerms_v2.xlsx): this includes revisions following Skype with Jim @bowring the morning of Wed 16 January (US Eastern Time). To explain:

The 4 possible permutations (P1-P4) for U-Th-Pb geochronology (according to SQUID 2.50), are defined in Columns B-I, rows 2-4. For each of these, columns C, E, G, I contain the current (v1.0.4) Squid3 expression-names for P1-P4 as provided by Jim, and columns B, D, F, H show the "flags" for what type(s) of data the expressions relate to (C = concentration RM, R = isotopic RM, U = unknown samples). I manually "gapped" all the lists (gaps are pale grey) to ensure that equivalent expression-names always occurs on the same row across P1-P4 (and I have audited those gaps to make sure they are sensible). Jim ordered the expressions first by Flag, and second by alphabetisation: column A contains an index number that can be used to recover any disruption of the original ordering of his list.

NEW COLUMNS (J-N)
Candidates I will explain later.
Verbose Name is my best attempt at a UNIQUE name for each expression, for the internal use of Squid3 (noting that the questions of what should appear as "building blocks" in the Expression Manager, and what should appear as "column headers" in the report-tables are separate questions that will both need careful consideration).
Data Affinity is an attempt to untangle the disparate expressions I have been asked to name according to where/how those expressions should be grouped/stored; I have placed them into 6 categories, 4 of which are Models.
Type corresponds to the "single-cell" flag in SQUID 2.50; the aim is to separate expressions that work on a row-by-row basis the majority) from those that generate "summary results" (on the RM) which generate vector output (corresponding to "arrays with labels" output in SQUID 2.50). How will "summary results" be reported back to users CSV-style?
Length corresponds to the fixed length of "summary" vector output (only).

CONVENTIONS
Syntax: I have deleted all hyphens and periods, all pre-existing incarnations of parentheses, and all asterisks except for those used to define Model Isotopic RM values (which theoretically originate from IDTIMS data anyway).
Prefix for aggregated "summary results": Several R-flagged expressions incorporate some sort of aggregation, and I have added a descriptor (Avg, BiWt, or WtdAv) to help those stand out. Hopefully self-explanatory!
Correction types: I have used Uncorr, 4corr, 7corr, 8corr universally to designate "SHRIMP-style" common Pb correction-types, and where multiple corr-types are present, the hierarchy descends from the left (unique names only; I hope we never display stacked corr-types to users!)
Nuclides: In isotopic ratios, I have specified numerator and denominator nuclides in full. I am open to the idea of dropping the element-symbols to make the names shorter (it probably would not decrease clarity); a question would be whether people are happy for element-labels to be missing in instances where the nuclide is NOT part of a ratio. For example, is "4corr Common 206 (%)" an acceptable alternative to "4corr Common 206Pb (%)"?
Units: Where the output of an expression is NOT dimensionless, I have suffixed the expression-name with its units in parentheses (these being the units that users will want to work with, not the units of the values Jim is actually storing!). I don't think these are superfluous: indicators like "(Ma)" or "(ppm)" might be self-explanatory, but I think a lot of the uses of "(%)" are informative.
Suffix: I did play with prefixes, but they were messy, so I have reverted to suffixes. But I have applied them to R-flagged expressions (in the form "RM"), rather than unknowns, in the hope that this will make expression-names easier to use in subsequent expressions. Yellow-coloured expressions are multi-context: the curly braces are intended to indicate that the suffix RM is used when wishing to refer to the relevant R-flagged expression, whereas no suffix is used when referring to the U-flagged one.

CATEGORY RU
The majority (20) of the 27 expressions labelled "RU" really pertain to data that should be held as part of Models (of various kinds: Concentration, RM, Isotopic RM, Default Common Pb, Physical Constants). Quite a few of them don't involve expressions as such: things like Concentration RM parameters and Physical Constants parameters are just numeric values with units.

The remaining 7 are all truly multi-context "built-in" expressions, so I have assigned names with an optional suffix as per above description of curly braces. These multi-context expressions have been retained, in preference to duplicating the expression as a single R-flagged expression plus a single U-flagged expression.

CATEGORY R
In general, I have on applied the suffix RM where it is needed.

Expression-names with white text on black background denote ALIASES as used in Squid3 v1.0.4, the exact expression for which is dependent on the index isotope chosen for the common Pb correction. The "Candidate" column contains a comma-separated list of Index numbers (i.e. column A) of longer/verbose/unique expression-names that are potential candidates for that alias. Note that many of the aliases apply only to P2 and P4, but some apply across all of P1-P4. Candidate index-numbers in square brackets apply to P1 only; all others apply to every white-on-black expression-name on that row.
I think and hope this alias-list (and alias-mapping) is complete for Squid3 v1.0.4, but I guess we will need to think more widely about aliases that might be desirable.

Note also that R-flagged expression-names essentially have two states: (1) "RM assessment", where pretty much every expression needs to be identified by its full unique name, prior to any choice being made about index isotope for common-Pb correction, and (2) "RM finalised", where that choice has been made, and useful aliases can be deployed fro both the RM and the unknown on that basis. I think we need to think carefully about whether we have a need to report state (1) to users in any form (i.e. application interface, or CSV-type report). Conceivably the answer is no: at present, one of the two(/three) radio-buttons 204corr/207corr(/208corr) is always selected, so it's possible that state (2) is all we need to show or report.

CATEGORY U
These are a bit simpler, and hopefully self-explanatory. Aliases as per category R.

TO DO
Brief definitions of each expression. I already have a fair bit of applicable material in hand (having written definitions of Oracle database fields designed to capture data from SQUID 2.50 workbooks), but there is some extra work to do.

Anyway, let me know what you think.

[bowring]

Great progress! I have a few initial considerations:

1. In keeping with the use of the other parameter model types, we should refer to "Common ..." and not "Default Common .."

2. We should not use (Ma) for Lambdas and Ages in the builtin expressions - I have gone to great lengths to make sure the underlying math follows the principles of separation of 'view' and 'model = data' so that the data is agnostic of units other than ground truth (annum). The place for (Ma) is in configuring the reports (as in ET_Redux), where the user selects the units for display of the underlying values - this functionality has not yet arrived in Squid3. The parameter model architecture inherited from ET_Redux follows this convention as well. All uses of lambda in the existing built-in expressions refer to lambda in annum and all of the underlying math in Ludwig's procedures has likewise been modified to work with annum. The use of (Ma)- tagged Lambdas will result in the existence of two sets of built-in lambdas. Thus the definition of ["Lambda 230Th (PerMa)"] will be ["lambda230"] / 1000000 and should really be considered a "custom expression" as it will only be used in custom expressions. Likewise, all calculations involving Age are designed to handle annum. If the user wants to view (Ma), they can specify the units in the report generator (yet to be implemented). The general idea of the report-customization engine will include the ability to create and display 'custom expressions' of any kind and in any units.

3. I am not sure there is an elegant solution, but "WtdAv Xcorr 206Pb/238U Calib Const" will not fall alphabetically with the other Xcorr expressions.

4. The {Contextual Suffix} = {RM} is not necessary because of the expression architecture: An expression knows the type of its valid targets (RM and/or U) and when invoked on an invalid target, will not yield a result. These yellow-highlighted expressions yield a result regardless of the type of target provided (RM, U) and are thus inherently context-sensitive and the user does not need to worry about it.

5. I have left out of this list the hard-coded parameters that appear at the bottom of the task audit and probably need better names:
   ExtPErr = 0.75
   L1033 = 1.033
   L859 = 0.859
   r238_235s = 137.88
   We should also decide if any of these parameters belong in our models.

6. Should we worry at this juncture about providing short explanations for each expression or simply rely on the the new verbose naming scheme for that info?

Cheers, Jim

[cwmagee]

Re: 3:
Can you customize your alphabetizer to ignore "WtdAv" in the same way that many modern alphabetizers ignore "the" and "a"?