-
Notifications
You must be signed in to change notification settings - Fork 910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
US-MISO has large negative 'unknown' value #1482
Comments
@corradio Can you extract the unknown time series for the last couple of days from the database and put it in a graph? or is the negative unknown value not recorded? Unknown category looks a bit like solar during the day, but might also be pumped storage (I feel like there was some info in the MISO issue back then... but I have no time to check atm) |
Unfortunately it's not recorded as it doesn't pass the validation :(
…On Thu, Jun 28, 2018 at 9:16 AM Alex ***@***.***> wrote:
@corradio <https://github.com/corradio> Can you extract the unknown time
series for the last couple of days from the database and put it in a graph?
or is the negative unknown value not recorded? Unknown category looks a bit
like solar during the day, but might also be pumped storage (I feel like
there was some info in the MISO issue back then... but I have no time to
check atm)
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<electricitymaps/electricitymaps#1482 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABlEKKNmRfzZD-_qrLgPVnjWYIGtvDNqks5uBILkgaJpZM4U60Ta>
.
|
Quoting the Miso site;
So pumped hydro? |
@systemcatch the negative part most likely comes from pumping, yeah. But then we should probably dismiss "other" as a whole, as pumping while generating will reduce the output of this category and therefore the emissions, too. |
This is still happening, still due to the negative "other" - mostly at night, supporting the pumped storage theory. We could write and ask if there is a separate source listing real-time pumping use, though scanning through their website I didn't see one - and https://www.misoenergy.org/markets-and-operations/RTDataAPIs/ has a quite extensive list already... Another idea would be to try to establish usual or average mix of what is in "other", to help us make the decision on what to do with it. If the maximum value of "other" is ~3 GW, this is in worst case around 5% of the total demand. It's probably not the worst approximation to just drop the "other" altogether. (From a CO2eq point of view, we have more inaccuracy from different kinds of coal anyway...) |
What about we update the parser and set to 0 negative values that are below 1% of total production (instead of looking at an absolute criterion, let's look at a relative one). |
Data point: from https://en.wikipedia.org/wiki/List_of_pumped-storage_hydroelectric_power_stations, https://en.wikipedia.org/wiki/Ludington_Pumped_Storage_Power_Plant is in MISO area (Michigan) with listed generating capacity of 2172 MW. That actually lines up kinda nicely with the values in charts in Bruno's post. Another idea: look through MISO website for historical reports as to fuel sources per year, and compare with what is listed in "other" |
Idea about simple possible solution: whenever "unknown"/MISO's "other" is negative, assign it entirely to hydro storage; when it's positive, assign it to "unknown". That is, when "other" is -1700 MW, that is possibly made up of say 200 MW run-of-river hydro, 100 MW diesel, and -2000 MW of hydro storage. Instead we assign the whole -1700 MW to hydro storage. Some data from yesterday (Friday before a long weekend): MISO peak load was just under 100 GW; in the morning once "other" returned to positive the low was around 65 GW. The highest value seen for "other" was 3.2 GW near 2pm local time when the load peaked. Upsides:
Downsides:
Thoughts? |
In terms of real-world physics, this looks fair enough, especially as there is less chance than average to have both storage AND other thermal (diesel, solid waste, garbage and wood pulp burners) at the same time. Storage is when prices are the lowest I'll let oli have a say about the backend data pipeline. |
Thank you for the extensive review @jarek.
As a principle, I tend to try to have reliable data in databases. This
means I strive to have data that is complete in the sense of:
- When looking at hydro data, it actually IS hydro (i.e. we *know* that
it's hydro)
- When looking at hydro data, we know it includes all of the hydro (i.e.
there's no other hydro that has been added to other categories)
I feel we're cutting too many corners here. The biggest problem is that at
some point we might consider the discharge intensity to be the averaged
intensity of stored energy. This might turn out to be very different than
the diesel/run of river average - hence I'm afraid we're making a mistake
by mixing them together.
I'm not satisfied with my own answer though, because on the other hand,
simply ignoring data when it's negative doesn't seem a good idea either.
Another suggestion is to simply truncate the unknown category (set to 0
when negative). This would bias the total load amount, but would not change
the mix. It would however assume that no storage is going on.
Finally, if we can argue that unknown is mostly storage, we should map all
unknown to hydro storage. If we can argue that unknown is mostly *not*
storage, we should consider it as such. However, it seems like this latter
assumption doesn't hold, as the value is largely negative.
If none of those cases can be argued for, then we have a problem.. I'll try
to think about it.
…On Sun, Sep 2, 2018 at 7:10 PM Bruno Lajoie ***@***.***> wrote:
In terms of real-world physics, this looks fair enough, especially as
there is little change to have both storage AND other thermal (diesel,
solid waste, garbage and wood pulp burners) at the same time!
I'll let oli have a say about the backend data pipeline.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#1482 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ABlEKBeLiiNrZ-HSf_JTFJ0-o-s4BS6Vks5uXBD3gaJpZM4U60Ta>
.
|
An alternative is to dig up statistical data on what actually makes up this "unknown" category over longer periods of time. MISO might have it on their website... Unfortunately I don't have time to go searching at the moment. Last couple of days MISO hasn't gone negative overnight, it looks like this as a sample: |
OK, happening again, e.g. currently -421 MW at 2:45 local time |
Closing due to inactivity. Thank you all for your inputs! |
The text was updated successfully, but these errors were encountered: