Validate loads and feeders

[nesnoj]

The loads and generators connected to MV and LV grids should be validated to make sure DINGO doesn't eat nodes: Check if the capacity of loads and feeders for every voltage level matches the data from OEDB.

[nesnoj]

One additional task: Check if data is consistent with transmission grid data - it is supposed to but we should to make sure nothing is lost in the whole (complex) process.

Checklist:

• Check if cumulative generation capacity (per voltage level and generation type) matches data from OEDB (tables supply.ego_dp_res_powerplant and supply.ego_dp_reconv_powerplant)
• Check if cumulative demand (per sector) matches data from OEDB (table demand.ego_dp_loadarea)
• (optional) Check if cumulative generation and demand data is consistent with transmission grid data on respective HV-MV substation.

Please include the DP version used for validation.

Implement this as a function (maybe multiple functions) in tools/validation.py.

[miguelparadacontzen]

I am having the following problem when comparing the peak_load information.

I have a network nw with some arbitrary MV districts saved in a pkl file.

If I query the LA info from the data base for the same districts, I obtain that

peak_load_agricultural     9831.968
peak_load_industrial      18746.509
peak_load_residential     31868.120
peak_load_retail          19666.418

If I loop over all the LAs that are effectively in nw (for LA in mv_district.lv_load_areas():), and rescue the attributes LA.peak_load_agricultural, etc, I obtain the same info. The ID attribute LA.id_db also coincide with the info I query from the data base.

However, if I loop over the LV districts within each LA (for lv_district in LA.lv_grid_districts():), I obtain a different information when adding all the attributes lv_district.peak_load_agricultural, etc :

peak_load_agricultural    10150.438
peak_load_industrial      18729.568
peak_load_residential     32271.766
peak_load_retail          19938.170

why?

I don't think is similar to this, because I'm using pandas to add everything together.

Edit:

if I further loop over the load nodes in each lv district
for node in lv_district.lv_grid.graph_nodes_sorted(): if isinstance(node,LVLoadDing0):
I obtain more non coherent numbers :)

[nesnoj]

Could you please provide your code snippet?

[gplssm]

Thanks for uncovering data inconsitency. I would be interested as well in the code. Maybe we go through quickly. Just to make sure no wrong comparison is made. Once we are sure the comparison is right, we can collect ideas how to find the cause and resolve it.

[miguelparadacontzen]

Code in here

to query the info from the db I use this

[nesnoj]

@gplssm Who's in charge?

[gplssm]

@miguelparadacontzen is continuing. Plan is as follows

1. Make sure peak loads in LA and LV grid districts tables are the same
2. Make sure these are the same at Ding0 lv_grid_district objects
3. Make sure cumulative peak load of all nodes is the same

If we don't have further ideas, we'll consult you... ;-)

[gplssm]

Comparing the cumulative peak load of LV grid districts to peak load of according load areas I found out these deviate significantly. But, this is already resolved in DP v0.3.0 which is available in "model_draft".

Versioned (currently used in Ding0)

diff_residential      92549.608809
diff_retail          276115.733138
diff_industrial      -77962.741536
diff_agricultural    338403.227488

model_draft

diff_residential      1.226731
diff_retail          -1.759502
diff_industrial      -5.175576
diff_agricultural    51.138129

both in kW.

Another detail popped up: Ding0 excludes LA's with very small (load_area_threshold = 1 kW). When calculating sectoral peak loads for LV grid districts in ego_dp_lv_consumption_peakload.sql this exclusion has to be done as well.

[gplssm]

The deviation of peak loads must be resolved in the data processing, therefore see this issue. I see this issue as resolved, when functions to check validity of data are ready.

[nesnoj]

Hmmm, but the deviation in the versioned one is less than in model draft - did u mix something up?

[gplssm]

Hmmm, but the deviation in the versioned one is less than in model draft - did u mix something up?

Yes, thanks for notifying me.

[nesnoj]

Yes, thanks for notifying me.

In the DP issue openego/data_processing#172 too...

[mltja]

Closing, issue is tackled for new data in #379