[Data Issue]: BD imports are counted as generation in source

[VIKTORVAV99]

When did this happen?

Ongoing since start?

What zones are affected?

BD

What is the problem?

It seems like the source is including imports as generation but we do not, so where are inferring an unknown production that is almost 1=1 with the imports. The parser needs to be updated to account for this and then all data needs to be re-fetched and reprocessed.

[q--]

Small thing I noticed after I pointed this out in #5381 (comment) is that, for a few data points, generation data is incomplete (e.g. a bit of gas generation appears to be unreported); in that case 'unknown' is not only imports. Otherwise it's all imports like you've described.

Screenshot

So the issue happened approximately 13:30-14:30 CET

(I don't know if it happens often or whether it's corrected automatically when data is re-fetched, I just wanted to point out that, sometimes, this happens.)

[VIKTORVAV99]

Small thing I noticed after I pointed this out in #5381 (comment) is that, for a few data points, generation data is incomplete (e.g. a bit of gas generation appears to be unreported); in that case 'unknown' is not only imports. Otherwise it's all imports like you've described.

(I can look for a screenshot for the day I noticed this if it helps. I don't know if it happens often or whether it's corrected automatically when data is re-fetched, I just wanted to point out that, sometimes, this happens.)

Any additional information would be helpful!
I also noticed that for some datapoints there is actually unknown production but I have only seen single digits.

If it's corrected in the source it will be corrected in our data as well :)

[q--]

I've added a screenshot to my comment

[VIKTORVAV99]

I have started refetching all the historical data now that this is deployed. But I won't re-reprocess everything because of this alone. There are some other changes that affects historical data as well coming soon though. Once those are merged (both on the contrib repo and backend) all historical data will be re-processed.