Creation of FSDP_process & revision of FSDP_collect

[mscrawford]

🐦 Purpose of this PR 🐦

This PR creates an intermediate step in the output generation for the FSEC project, FSDP_process. This is necessary because external researchers need results from the FSEC scenarios, and the FSDP_collect function can only fully execute once these datasets have been processed and the researchers' outputs returned. Therefore, this function produces these intermediate outputs. Afterwards, this PR adds functionality to the FSDP_collect function, so that it is able to read these external researchers' outputs and integrate them into the FSEC datasets (and subsequently the FSDP plots).

🔧 Checklist for PR creator 🔧

• Labeling pull request correctly from the label list.

• Low risk : Simple bugfixes (missing files, updated documentation, typos) or Start/output scripts
• Medium risk : New realization / Changes to existing realization / Other changes which don't modify default.cfg
• High risk : New input files (if cfg$input is changed in default.cfg) / Modification to core model (eg. changes in equations, calculations, introduction of new sets etc.) / Other changes in default.cfg

• Providing additional information based on PR label

• Low risk : No new model run needed.
• Medium risk : Default run based on the current version of the fork from which PR is made
• High risk
  • Default run from the current develop branch
  • Default run based on the current version of the fork from which PR is made

• 📉 Performance loss/gain from current default behavior 📈

  • Current develop branch default : ** mins
  • This PR's default : ** mins

• Added changes to CHANGELOG.md

• Compilation check (model starts without compilation errors - use gams main.gms action=c in model folder for testing).

• No hard coded numbers and cluster/country/region names.

• The new code doesn't contain declared but unused parameters or variables.

• Where relevant, In-code comments added including documentation comments.

• Made sure that documentation created with goxygen is okay (use goxygen::goxygen() for testing).

• Changes to magpie4 R library for post processing of model output (ideally backward compatible).

• Self-review of my own code.

• In case of updated cellular input tgz file in default.cfg: scenario_config.csv has been updated accordingly (rcp1p9, rcp2p6 etc)

• For high risk runs: validation of major model indicators - Land-use, emissions, food prices, Tau. %Delete this line in case it is not a high risk run%

⚠️ Additional notes or warnings ⚠️

NA

🚨 Checklist for RSE reviewer 🚨

• PR is labeled correctly.
• CHANGELOG is updated correctly
• No hard coded numbers and cluster/country/region names.
• No unnecessary increase in module interfaces
• All required updates in interfaces (if any) have been properly adressed in the module contracts
• In-code comments and documentation comments are satisfactory.
• model behavior/performance is satisfactory.
• Requested changes (if any) were applied correctly

🚨 Checklist for MAgPIE reviewer 🚨

• PR is labeled correctly.
• CHANGELOG is updated correctly
• No hard coded numbers and cluster/country/region names.
• Changes to the model are scientifically sound
• In-code comments and documentation comments are satisfactory.
• model behavior/performance is satisfactory.
• Requested changes (if any) were applied correctly

[bodirsky]

in principle, i recommend using our write.magpie and write.report functions as they have a more standardized format. but now, i would keep it as is.

[mscrawford]

The difficulty is that I need to run csv2gdx on it, so the .mif format would have been clunkier. Unless write.report can do more than .mifs?

[tscheypidi]

wouldn't it be an option to use writeGDX if the aim is to write the data to a gdx file?

[mscrawford]

I slept on this suggestion, and I think that - although it would greatly improve this entire workflow - I'm going to say that for now I would prefer to leave the code as-is. To do this properly, I would need to re-write the magpie4 function in addition to this function. When we port this code to future projects, I'll do the work of revising it properly. For now we have a working solution. I feel quite silly for not using this in the first place. I guess way back then I wanted to use .csvs because I thought it would be simpler for me and then got locked-in.

[tscheypidi]

Just some suggestion, but looks ok overall