Introduce more consistent par/obs names

[briochh]

I think it would be good to try and get the par (and maybe obs) naming conventions a bit more consistent. So decide whether we automatically add multiplier, direct, grid, zone, pp, etc and if we do make sure we do it consistently across all permutations.

Appreciate that I am currently raising more issues than I am fixing -- these are as much notes for me. But.... if anyone else wants to tackle, PRs welcome!

[jtwhite79]

Agreed. I was even thinking par_type and par_style could be name attributes with the colon delimiter? Maybe also file_style (array or list)?

[wkitlasten]

Sort of related (auto names) and more specific than necessary.

0. the name "mult2model_info.csv" maybe a little misleading since "direct" pars are also included in there. Maybe something like "pars2model_info.csv"?
1. "original" rather than "org" directory may be clearer? Also "multiplier" rather than "mult" directory?
2. not sure where the "inst" comes from, but it is "inst1pp" for my pp filenames and "inst0_zone" for my zone filenames, both acting on the same array... so a little redundant.

Maybe I could dig around and change some of this, but I wouldn't want to change it and risk breaking everyone's workflow/scripts without consensus.

[jtwhite79]

That last one is the kind of consistency thing I'd like to clear up. We still need the "inst" tag but would nice to have all of the name attributes with the "name:value" style so that we can parse them into Metadata columns...

[jtwhite79]

Well....you could use the new pst.rename_obseevations() method to have finer grained control over the names. I typically scrub those header and indices to deal with this problem. @briochh any thoughts on this?

[wkitlasten]

Sorry, I was in the middle of trying to explain and Alt+Tab+<twitch>+<glitch>...Entered my way into closing the issue and submitting a partial thought (again)! My point was "_" seems pretty common in table headers, parameter names, filenames etc. so using it to separate metadata fields seems potentially problematic. @briochh mentioned maybe using "-" as the metadata separator. Obviously there are many options, one that is less likely to be found elsewhere seems like a good choice. I will check out the new rename_observations() method. Thanks.

…

On Wed, Oct 6, 2021 at 1:45 PM J Dub ***@***.***> wrote: Well....you could use the new pst.rename_obseevations() method to have finer grained control over the names. I typically scrub those header and indices to deal with this problem. @briochh <https://github.com/briochh> any thoughts on this? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#280 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADSJXRBUBOFWXSGDNRI6P4DUFOLZVANCNFSM5DPFV3PA> .

-- "Perfect spheres are pointless."

[briochh]

General and safe storage of metadata components in par and obs names is a bit of a challenge. @wkitlasten is right, "_" is fairly ubiquitously used a space fill in model input and output file column headers. I haven't hit this issue myself but can see how it might come up.

@jtwhite79 would PEST and PEST++ handle "-" in par and obsname strings? If so perhaps we could migrate to using this as the metadata sep? It's not the complete solution (like what happens when someone has a "-" in there one of their requires columns) but may reduce the occurance of issues.

This is really all just to make it easy for pyemu to automagically strip out metadata into pst.parameter_data and pst.observation_data columns... not an imposibly task to do manually, if your strings are a bit quirky.

[jtwhite79]

@briochh - the dash should be fine and @wkitlasten is right that this is an issue - Ive hit it also. was just looking at my keyboard for a good general choice for metadata sep and I cant find one. Could go with double underscore or double sep? maybe a vertical bar? We could make it an arg for PstFrom but then we need to somehow feed that choice into Pst.try_parse_metadata(). Maybe we could add it as a ++ arg and Pst could detect it?