Handle TFM_INS variants

[siddharth-krishna]

These variants essentially allow specifying multiple attribute values in a row by un-melting a field (e.g. attribute, year) and allowing them to be column names. So to handle them, we find those columns, and melt it back into 2 columns: attribute & value (or year & value).

• TFM_INS-TS:
  • TS: is timeslice. Columns are years, should be melted into Year
• TFM_INS-AT:
  • AT: is attribute. Columns are attribute names, should be melted into Attribute
  • All valid attribute names are to be obtained from times-info.json (everything with category = parameter)
  • @olejandro I don't see any tables tagged TFM_INS-AT in any of our benchmarks. What should we use to test this?
• TFM_INS-TXT:
  • @olejandro do you know how we should handle this one?

@olejandro I'm afraid I'll miss the meeting today, my update is that I'm working on this feature, and need some guidance with the questions above. I've implemented TFM_INS-TS but now I'm fixing some regression bugs.

[olejandro]

Thanks @siddharth-krishna. the TXT variant should be handled differently. #84 focuses on it.

[olejandro]

Probably the easiest way to test this is to modify an existing benchmarks. Should I do it?

[olejandro]

We should also add TFM_INS-TSL to the list. Here columns are timeslices.

[siddharth-krishna]

Modifying one of the demos sounds good! Will be easier to debug :)

[olejandro]

Ok, I've changed the fifth one...

[Antti-L]

TFM_INS-TS:
* TS: is timeslice. Columns are years, should be melted into Year

No, I believe TS stands for time series. That is why the columns are years in this case, because this format is meant for more convenient defining of time series parameters with many data points (years). Timeslice is in this format specified in the Timeslice column, as usual. Another place where TS stands for time series is the TS_Filter column, which can be used in the TFM_INS variants, and refers to time series filtering.

[olejandro]

Thanks @Antti-L. I've overlooked the typo. @siddharth-krishna please notice the difference between tfm_ins-ts and tfm_ins-tsl. We probably should add a test case for the latter...

[Antti-L]

These variants essentially allow specifying multiple attribute values in a row by un-melting a field (e.g. attribute, year) and allowing them to be column names.

BTW, I hope you are aware that ~FI_T tables also allow using Years as value column headers, in addition to e.g. using Attributes as value column headers. I am using such flexible layouts all the time in my models, and I find it very convenient. I guess FI_T in fact stands for "flexible input table", and the flexibility is indeed very nice.

[olejandro]

Thanks @Antti-L. We should open a separate issue on ~FI_T tables if they are not handled correctly.