Add parameters list file produces tpl file that seems to be incompatible with input file

[aymanalz]

pf.add_parameters(filenames= "sfr_cond.csv", use_cols = ["cond_value"], par_type='grid', index_cols=['ZoneID'],
par_name_base="sfr_", pargp="sfr_cn", upper_bound = 1.5, lower_bound=0.5)

The parameter input file "sfr_cond.csv" is

Unnamed: 0 | parnme | cond_value | ZoneID
0 | sfr_zone_0 | 9.879605293 | 0
1 | sfr_zone_1 | 0.003133586 | 1
2 | sfr_zone_2 | 0.138467833 | 2
3 | sfr_zone_3 | 2.117573261 | 3
4 | sfr_zone_4 | 0.042645961 | 4
5 | sfr_zone_5 | 8.667610168 | 5
6 | sfr_zone_6 | 20.70874214 | 6
7 | sfr_zone_7 | 23.52819443 | 7
8 | sfr_zone_8 | 3.487426519 | 8
9 | sfr_zone_9 | 18.8850708 | 9
10 | sfr_zone_10 | 6.028618336 | 10

the resulting tpl file is
ptf ~
,sidx,idx_strs,xy,x,y,pargpcond_value,parval1_cond_value,cond_value
0,"(-1,)",ZoneID:-1,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:-1 ~
1,"(0,)",ZoneID:0,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:0 ~
2,"(1,)",ZoneID:1,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:1 ~
3,"(7,)",ZoneID:7,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:7 ~
4,"(2,)",ZoneID:2,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:2 ~
5,"(8,)",ZoneID:8,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:8 ~
6,"(3,)",ZoneID:3,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:3 ~
7,"(9,)",ZoneID:9,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:9 ~
8,"(4,)",ZoneID:4,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:4 ~
9,"(5,)",ZoneID:5,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:5 ~
10,"(6,)",ZoneID:6,"(None, None)",,,sfr_cn,1.0,~ sfr__inst:0_usecol:cond_value_ZoneID:6 ~

[briochh]

Hi @aymanalz,

This doesn't look horrific, the tpl files won't necessarily related to model input files, as they are not mapped to model input files until the apply_mult() methods are called.

Is there are specific error that you are getting when you run PEST or the apply_mult() methods?

[jtwhite79]

I think that tpl is for the multiplier process. Can you try using direct-style pars for this and see how the tpl file turns out (Im not sure about that vertical bar sep tho...)

[aymanalz]

Thank you @briochh and @jtwhite79 for your quick reply! In fact I did not apply_mult(). I was checking the generated files after I added new set of parameters using pf.add_parameters().
I tried the direct-style and it does what I need. I noticed two things:
(1) if we use index_cols that has a field name with upper case, an error will occur. For example index_cols=['Zone'] will not work but index_cols=['zone'] will work. This is the exception "Ensemble._gaussian_draw() error: the following cov names are not in "
"mean_values: {0}".format(",".join(missing))

(2) The generated template file for a direct style does not have the header of the input file. For example

input file

Unnamed: 0,parnme,cond_value,zone
0,sfr_zone_0,9.879605293273926,0
1,sfr_zone_1,0.003133586375042796,1
2,sfr_zone_2,0.13846783339977264,2

tpl file

ptf ~
0,sfr_zone_0,~ sfr__inst:0_usecol:cond_value_direct_zone:-1 ,0
1,sfr_zone_1, sfr__inst:0_usecol:cond_value_direct_zone:0 ,1
2,sfr_zone_2, sfr__inst:0_usecol:cond_value_direct_zone:1 ~,2

I was expecting something like
ptf ~
Unnamed: 0,parnme,cond_value,zone
0,sfr_zone_0,~ sfr__inst:0_usecol:cond_value_direct_zone:-1 ,0
1,sfr_zone_1, sfr__inst:0_usecol:cond_value_direct_zone:0 ,1
2,sfr_zone_2, sfr__inst:0_usecol:cond_value_direct_zone:1 ~,2

By the way the tool is amazing!!

[aymanalz]

Please ignore vertical lines and dashed lines they occur as i copy from csv file opened in excel.

[briochh]

That missing header sounds familiar. It was possibly related to #184 and hopefully fixed in #187. Are you using the latest on develop? If you are then will need to look into it a bit more.

[aymanalz]

Yes, I am. From the commits log, it seems that the last commit was "Merge pull request #182 from jtwhite79/feat_io_df"

[briochh]

Should be some updates from then. Give it a quick git pull and see how that goes.

[aymanalz]

Problem solved! I was working from a fork that is not in sync with main repo. Thank you!

[briochh]

No problem @aymanalz. Keep reporting issues you find!