Reformat agent parameters in example model

[tsmbland]

Description

Reworks the example model to fit the new schema provided by @ahawkes, particularly in relation to agent parameters. See schema

Obviously this cannot be merged yet as the example model will not run. The plan is to build on top of this branch, and merge into main only when everything is ready (or sufficiently ready so that the example model runs without error).

A couple of questions/comments:

• We could combine agent_parameters.csv into agents.csv as these are 1 to 1. What do you think?
• Are the options for agent region just a single region or "all"? (whereas before, any set of regions could be specified for each agent)
• We don't yet have the option to put "all" in the year field (this applies to other parts of the code too, see [UMBRELLA] Allow "all" option for year and region #444), so for now I've copied data across all years where necessary (e.g. agent_objectives.csv)

---

To help guide development, I'm also listing the validation steps that I think we'll need to perform on the data:

agents.csv

• each region must have at least 1 agent

agent_parameters.csv

• for each agent there must be exactly 1 row
• if decision rule is lexico, lexico tolerance needs to be provided

agent_commodities.csv

• for each agent there must be at least 1 commodity
• for each agent/commodity combination all years must be specified
• commodity portions must to sum to 1 for each commodity in each year (presumably at the region-level rather than globally? so this would need to check the region of each agent)
• all commodities of SVD or SED type must be covered

agent_cost_limits.csv

• only permit agent/commodity combinations specified in agent_commodities.csv
• if an agent/commodity combination is specified for one year, it must be specified for all years (at least for now)
• (if an agent/commodity combination is not specified, no limit is applied)

agent_objectives.csv

• at least one objective must be specified for each agent
• all years must be covered for each agent
• if agent decision rule:
  • single: only one objective can be specified for each year
  • weighted: all weights must sum to 1 for every year
  • lexico: decision_lexico_order must be specified for all objectives, and all objectives must have a different order

agent_search_space.csv

• only permit agent/commodity combinations specified in agent_commodities.csv
• if an agent/commodity combination is specified for one year, it must be specified for all years
• (if an agent/commodity combination is not specified, the "all" search rule is applied)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.51%. Comparing base (819da65) to head (63e148b).
Report is 253 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #442      +/-   ##
==========================================
- Coverage   95.31%   86.51%   -8.81%     
==========================================
  Files          31       31              
  Lines        4272     4272              
  Branches     4272     4272              
==========================================
- Hits         4072     3696     -376     
- Misses        105      497     +392     
+ Partials       95       79      -16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[tsmbland]

@alexdewar

[alexdewar]

A couple of questions/comments:

• We could combine agent_parameters.csv into agents.csv as these are 1 to 1. What do you think?

I think this will make things a bit tidier. (We currently have a separate process_parameters.csv which is also 1:1 with processes.csv and I don't really like that, but my understanding was that at some point we will want processes with multiple parameter sets.)

Side note: for decision_lex_tol and decision_weight, as we only need them under certain conditions, I think the idiomatic way to express that would be to have these values inside the relevant enum(s) rather than as Options (it's a bit safer and also makes it clear which values are associated with the different enum variants). We haven't done that yet, but it's maybe something to bear in mind for when you're fiddling with this part of the code.

• Are the options for agent region just a single region or "all"? (whereas before, any set of regions could be specified for each agent)

Agent regions are represented with a RegionSelection, so you can have one or more region (separated by semicolons in the CSV file) or specify all.

• We don't yet have the option to put "all" in the year field (this applies to other parts of the code too, see Allow "all" option for year #444), so for now I've copied data across all years where necessary (e.g. agent_objectives.csv)

Ok cool. I think this is the right way to do things for now.

[ahawkes]

A couple of questions/comments:

• We could combine agent_parameters.csv into agents.csv as these are 1 to 1. What do you think?

Could do. I guess in future we may want decision rules to change over time, which would need its own table and more consistency checks.

• Are the options for agent region just a single region or "all"? (whereas before, any set of regions could be specified for each agent)

The default should be "all" if nothing is specified for region. Also would be good to accept "ALL".

• We don't yet have the option to put "all" in the year field (this applies to other parts of the code too, see Allow "all" option for year #444), so for now I've copied data across all years where necessary (e.g. agent_objectives.csv)

Where year is not specified we should assume it applies to all years. And yes could also accept "ALL"

To help guide development, I'm also listing the validation steps that I think we'll need to perform on the data:

agents.csv

• each region must have at least 1 agent

agent_parameters.csv

• for each agent there must be exactly 1 row
• if decision rule is lexico, lexico tolerance needs to be provided

agent_commodities.csv

• for each agent there must be at least 1 commodity
• for each agent/commodity combination all years must be specified

As per the data schema, we should also accept nothing being entered for year, with the assumption that the value applies to all years. This behaviour would need to be documented. We should also check the user has not entered conflicting values for any year.

• commodity portions must to sum to 1 for each commodity in each year (presumably at the region-level rather than globally? so this would need to check the region of each agent)

Yes at region level for now. But note that when commodity is tradable between regions this will need to be different (as per MUSE1)

• all commodities of SVD or SED type must be covered

agent_cost_limits.csv

• only permit agent/commodity combinations specified in agent_commodities.csv
• if an agent/commodity combination is specified for one year, it must be specified for all years (at least for now)

As above, can we allow blank for year which implies all years?

• (if an agent/commodity combination is not specified, no limit is applied)

agent_objectives.csv

• at least one objective must be specified for each agent

• all years must be covered for each agent

• if agent decision rule:

  • single: only one objective can be specified for each year
  • weighted: all weights must sum to 1 for every year
  • lexico: decision_lexico_order must be specified for all objectives, and all objectives must have a different order

For lexico I think we also only allow only two levels for now. primary and secondary.

agent_search_space.csv

• only permit agent/commodity combinations specified in agent_commodities.csv
• if an agent/commodity combination is specified for one year, it must be specified for all years

As above, no entry means all years?

• (if an agent/commodity combination is not specified, the "all" search rule is applied)

Yes, and document this.

[tsmbland]

Thanks @ahawkes and @alexdewar for the comments!