Repository navigation
Missing Data in LMPs by Resource Nodes, Load Zones and Trading Hubs (NP6-788-CD) #133
|
Hi everyone, I’m currently working with the “LMPs by Resource Nodes, Load Zones and Trading Hubs” (NP6-788-CD) dataset and noticed several gaps in the historical data returned from the ERCOT API. 1. Missing historical intervals – March 6, 2026The dataset should update every 5 minutes, but the data available during the 3:00–4:00 PM CST window appears incomplete. The only records present during that hour are:
This suggests a large gap where multiple expected 5-minute intervals are missing. Additionally, it looks like most corrupted data for March 6 start missing around 4:55:18 AM CST, which also seems unusual given the normal cadence of the dataset. 2. Missing data today – March 9, 2026For today, querying the ERCOT API returns no data between 00:01:00 and 06:00:00 CST. It’s unclear whether this is due to a publishing delay, an API issue, or if the data has not yet been backfilled. (Only data on 2026-03-09T00:00:18, none past that time until 6AM) Data after 06:00:00 has already been published and is continuing to update in normal 5-minute intervals, which suggests this likely isn’t a simple delay or backfill scenario. Has anyone else encountered this issue with the NP6-788-CD dataset? Also, if there is a more reliable or canonical way to access complete LMP data (rather than resorting to scraping the ERCOT website), I’d appreciate any guidance on where to obtain a fully consistent historical dataset. Additional Question: Possible Changes to ERCOT API Rate LimitsSeparately, I’ve also noticed something that might indicate a change in the API rate limits. Previously, I was able to make roughly the documented ~30 requests per minute without issue. However, since around March 6, the API seems noticeably more restrictive, and I hit throttling much sooner than expected. I noticed this issue being brought up in #132 as well. It’s possible this is just imagination on my end, but I wanted to ask: Have the rate limits for the ERCOT API been adjusted recently? Are there new undocumented limits or enforcement mechanisms in place? Is there an updated recommendation for request pacing when querying large historical datasets? Any clarification would be greatly appreciated. Thanks in advance! |
Replies: 3 comments 1 reply
|
Hi, I am also experiencing the same issue as the one stated above. Would appreciate a quick response and resolution |
|
I am also experiencing issues with missing data for several endpoints which seems related to this question. Initially I thought this may be DST related, but I am seeing issues on days besides 3/7
Other third parties have data for these dates. Is it available somewhere besides these API endpoints? |
|
Hello seanxjin, The issue with the missing intervals for NP6-346-CD, NP6-788-CD, and NP4-190-CD has been resolved. All intervals should be present for March 6th, 7th, and 9th. There are several other artifacts that are currently processing in the queue as well (including NP6-905-CD) and will be available shortly. The issues we encountered on March 6th had prevented the processing of these files, but were automatically added to the queue and are being processed along with current report artifacts. We have also adjusted the base bandwidth limits in an attempt to alleviate some of the rate limiting issues users are currently experiencing. Regards, |
Hello seanxjin,
The issue with the missing intervals for NP6-346-CD, NP6-788-CD, and NP4-190-CD has been resolved. All intervals should be present for March 6th, 7th, and 9th. There are several other artifacts that are currently processing in the queue as well (including NP6-905-CD) and will be available shortly. The issues we encountered on March 6th had prevented the processing of these files, but were automatically added to the queue and are being processed along with current report artifacts.
We have also adjusted the base bandwidth limits in an attempt to alleviate some of the rate limiting issues users are currently experiencing.
Regards,
mackermann-ercot