Integrate tyndp 2020

[grecht]

Related to #57 .

Changes proposed in this Pull Request

• Added the option to include projects from the TYNDP 2020.
• Commissioning dates are included. Every non-TYNDP asset gets assigned the commissioning date 1970-01-01.
• The new configuration options include a possibility to set the desired project status and ignore selected projects.

In an external repository "tyndp_to_pypsa", coordinates were added to the TYNDP projects by hand based on asset names, the rudimentary maps on the TYNDP website, and OpenStreetMap. The data is converted there to the gridextract format so that it can be imported seamlessly into PyPSA-Eur. It is already split into asset type and "new" or "upgraded" based on the latest gridextract and links_tyndp.csv - but this means every time this data changes, the TYNDP files in data/tyndp2020 would have to be rebuilt using tyndp_to_pypsa.

To test this, follow the usual config file workflow (remember to set include: true in the TYNDP2020 options).
I have not tested this beyond building the base network (see plot at bottom).

Checklist

• I tested my contribution locally and it seems to work fine.
• Code and workflow changes are sufficiently documented.
• Changes in configuration options are added in all of config.default.yaml, config.tutorial.yaml, and test/config.test1.yaml.
• Check how capacities overall have changed as @martacki did here
• The TYNDP project status is thrown away, and instead under_construction=True is set in order to conform to the current format. Does this make sense?
• Is the way asset updates are applied problematic? In the current state when e.g. a line's capacity is upgraded, its voltage is updated but as it is a TYNDP project, under_construction=True is set. I do not know the implications of this, but the updated asset might not remain usable until the commissioning date of the update.
• It is unclear how to represent consecutive upgrades of the same asset: Right now, the "best" upgrade is taken, but this is also the latest with a more uncertain status. Probably does not occur often, but I did not check.
• Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
• A note for the release notes doc/release_notes.rst is amended in the format of previous release notes.

[pz-max]

CI is failing at the moment. Probably the test config needs adjustments.
Please test locally and add suggested fix.

[pz-max]

better -> allowed_status

[pz-max]

Maybe add here the README tyndp as docstring

[martacki]

CI should be passing now 🥳

problem: s_nom=0 \forall TYNDP lines.
if we assume num_parallel=1 for all TYNDP lines, and type=Al/St 240/40 4-bundle 380.0, then the additional build TWkm capacity is 10.94953991945518, i.e. 3-4% more than the existing grid. These assumptions are currently not part of the PR.

Wasnt TYNDP more like 25%, or did that include HVDC links? (we only consider lines here) @FabianHofmann @nworbmot

[martacki]

Quick update (probably mostly for my future self):
I tried running the whole workflow now, but resulting in non-intuitive behaviour, e.g. network can now be clustered to 42 nodes (not 37), where Germany and France now have 3 nodes (instead of 1), despite all nodes belonging to the same synchronous zone, which is strange.
norway NO now has 4 synchronous zones (instead of 1), and some additional buses that are not being aggregated but remain disconnected from the network (at a 42 node resolution), similarly for Sweden SE (2 synchronous zones now). Regardless, the aggregation connects all nodes and puts the extra 4 nodes to Germany DE and France FR, from my understanding.
The last additional nodes goes to Corsica/Sardinia, which from the algorithm perspective works as intended (because there are no links to be simplified). However, I believe that there is some errors with the new TYNDP links connecting the isles. In this case, the TYNDP states that the existing link is being upgraded, but there are 2 new links in this PR, while the existing ones are all being kept.
new links: TYNDP2020_29 + TYNDP2020_30

[p-glaum]

Here are some results from the validation runs comparing networks with and without tyndp2020 projects:

[p-glaum]

There was one mistake in the line filtering which kicked out many tyndp lines in the base_network script, which @martacki is fixing.
Below, we compare the pypsa-eur grid with tyndp 2020 to the project map from tyndp. There are still lines missing when comparing to tyndp 2020, but most of them are considered in this PR.
Next to ToDo is to rerun the pypsa-eur workflow and do the plots from above again.

https://tyndp2020-project-platform.azurewebsites.net/projectsheets

[fneum]

@martacki @p-glaum, what was the latest status for this PR? Do you think this is ready to be merged?

[p-glaum]

not yet, still need to do some runs on the cluster to validate this PR

[p-glaum]

Hi @grecht,
could you maybe also give me contributor rights that I can make adjustments to your code? Then I do not always have to ask @martacki.