[Data Outage]: Missing data from ENTSO-E for Germany

[unitrium]

Note

This issue is used for continuous data outages for Germany since the start of 2024!

Update Feb 22 2024: All data is back and has been refetched, but a new week of maintenance mode has been announced by TSOs publishing to ENTSO-E, so we are expecting more outages and incorrect data until 28th of February.

Update Feb 18 2024: We have been missing data on/off for over a week, we are monitoring the situation closely, but we cannot do much else at this stage.

---

Previous description

When did this happen?

Seems to have started sometime around 2024-01-31 13:00 (last timestamp have in DB)

EDIT: Since 2024-02-02 Amprion is publishing 0 values to ENTSO-E which causes a reduction in the solar values for Germany agains expected values.

What zones are affected?

DE

What is the problem?

Amprion is unable to collect solar data and will since 2023-02-02 publish 0 values for technical reasons.

From ENTSO-E

Dear Transparency Platform users,

Since 02.02. Amprion cannot provide data for the publication of the energy carrier Solar of publication 16.1.B/C. For technical reasons, 0 values will therefore be published temporarily.

As soon as the problem is solved, Amprion will update the correct values.

Best regards,
Transparency Platform Team on behalf of Amprion

Once this is resolved we will update the values.

[unitrium]

Funny enough the data is not present for the bidding zone but is present on the control area level.
Will monitor the situation and refetch when available.

[w-flo]

Only Amprion data seems to be missing, it's listed as "N/A" on the transparency platform.
For the country-level sum, X + N/A = N/A though.

Offshore wind data is available, because Amprion's control area has no offshore wind farms (those are all TenneT or 50Hertz).

This happens somewhat regularly. Seems like every couple of months, one of the German TSOs has some data reporting issues.

[w-flo]

Data seems to be available now. (Edit: I meant Amprion data. The country sum data at Entso-E transparency platform is still N/A for some reason.)

[VIKTORVAV99]

@unitrium should we close this?

[unitrium]

Let's refetch the data and reflowtrace before closing

[VIKTORVAV99]

Let's refetch the data and reflowtrace before closing

That has already happened automatically, it was missing less than 24 hours so the fetching already got the data (-72 hours) and the flow tracing should also have caught it (-48 hours).

[unitrium]

Ok so let's close it now, I will remove the outage message as well.

[w-flo]

I still see the outage message for some reason, maybe it's slow to update?

Edit: And it's gone. Thanks!

[madsnedergaard]

Data has been missing since 2/2, reopening this issue - it's Amprion again :(

[Fritzboing]

Wann ist mit einer Lösung zu rechnen?

[VIKTORVAV99]

Wann ist mit einer Lösung zu rechnen?

Should be fixed the next hour now that there is data in ENTSO-E but we are monitoring the situation.

[madsnedergaard]

Wann ist mit einer Lösung zu rechnen?

Should be fixed the next hour now that there is data in ENTSO-E but we are monitoring the situation.

Have there been any news on why data was missing or did you just check regularly?

[VIKTORVAV99]

Wann ist mit einer Lösung zu rechnen?

Should be fixed the next hour now that there is data in ENTSO-E but we are monitoring the situation.

Have there been any news on why data was missing or did you just check regularly?

I just have the page up in the background and refresh it every hour when I'm at the computer 😅

[VIKTORVAV99]

Should be resolved now though, but let's leave the outage message up as I don't think we have flow traced the earliest missing data yet.

(and monitor so they actually release new data from now on and it's just not a one off upload)

[VIKTORVAV99]

We have gotten more information from ENTSO-E about this issue so I have updated the description and will keep this open for now.

[Spanni26]

Hmmm, energy-charts seems to have data for that period. Are they using a differnet source?

[VIKTORVAV99]

Hmmm, energy-charts seems to have data for that period. Are they using a differnet source?

We are not missing any data but the data published to ENTSO-E is incomplete for solar at the moment. Apparently Amprion is publishing 0 production for solar due to a technical issue.

[madsnedergaard]

Hmmm, energy-charts seems to have data for that period. Are they using a differnet source?

Energy Charts have some really great fallback models in case real-time data is missing - as we cover a lot of countries we unfortunately don't have the resources to make such detailed estimation models for each country yet :)

[madsnedergaard]

Sigh, it feels like we should just keep this issue pinned and always open 😓

Over the last week or so, there's constantly been problems with the data for Germany... Now it seems like it's TransnetBW that is not reporting any data.

[madsnedergaard]

Latest update from ENTSO-E

Dear Transparency Platform users, On 27th February 2024, TenneT DE will perform major updates in its publication tools which might cause delayed or missing publications of any data item. TenneT DE will monitor the publications and resend missing data as soon as possible. Thanks for your understanding! Kind regards,Transparency Platform team on behalf of TenneT DE

[cgicgi]

Translation of the above message from ENTSO-E:
"We and our partners have serious issues for weeks now providing correct data. To make things worse, another one of our partners will cause new issues by the end of the month to make you forget about the seriousness of the already existing issues. Mind the gap"

[VIKTORVAV99]

Now we seem to be getting some data but it's all a mess of missing or reduced values.
I'll take a look and see if there is anything we can do on our side but on the top of my head I don't think there is.

[madsnedergaard]

We have pretty good data back as far as I am aware, but ENTSO-E has warned of maintenance from TSOs between 23/2 and 27/2. I will adjust the warning message to make that more clear.

[VIKTORVAV99]

There was some missing data and bad estimations but I refetched and reprocessed those hours so should be all good now and it will be corrected for the daily data in the app in a few hours.

[w-flo]

Solar "actual generation per production type" data is still missing for Amprion from 2024-02-03 to 2024-02-05.

[VIKTORVAV99]

Solar "actual generation per production type" data is still missing for Amprion from 2024-02-03 to 2024-02-05.

Not much we can do about this I'm afraid, not until it's corrected in the ENTSO-E transparency platform.

[w-flo]

Sorry! I should have stated explicitly that I was refering to the ENTSO-E transparency platform. They're still missing solar data for Amprion from Feb 2 to Feb 5.

[w-flo]

Hooray! Amprion has now published solar data for early February that was still missing a few days ago. I think the data is now complete on Entso-E transparency platform?

Let's hope today's TenneT maintenance won't have any negative effects.

[VIKTORVAV99]

Hooray! Amprion has now published solar data for early February that was still missing a few days ago. I think the data is now complete on Entso-E transparency platform?

Let's hope today's TenneT maintenance won't have any negative effects.

I'll make sure to refetch and re-process the data. 🎉

[VIKTORVAV99]

Refetched the data and started reprocessing, things should be updated on both the API and app by tomorrow.