Secure EWS historical backfill: short request window for RUC/state-estimator products?

[iamfaith99]

Hi ERCOT/API team,

I am working through historical backfill for several Secure EWS CSV report products and I am trying to understand the expected request-window behavior. Recent/current pulls work, but when requesting older history the available result set appears to be limited to a relatively short recent window or returns sparse/no documents for earlier operating dates.

The products I am trying to backfill are:

EMIL ID	Report Type ID	Product	Frequency	Notes
NP3-217-CD	12346	Forecasted Temperature Adjusted Dynamic Ratings	Chron - Hourly	Secure / ECEII, high-row-volume feed
NP5-523-CD	13108	Daily RUC Deselected Capacity	Event - Per DRUC Run	Secure RUC data
NP5-524-CD	13109	Hourly RUC Deselected Capacity	Event - Per HRUC Run	Secure RUC data
NP5-757-CD	13080	Deselected Daily RUC Recommendations	Event - Per DRUC Run	Secure RUC data
NP6-627-CD	13048	State Estimator Bus Voltages	Chron - Hourly	Secure / ECEII, high-row-volume feed
NP6-93-CD	12304	State Estimator Switch Status	Chron - Hourly	Secure / ECEII, high-row-volume feed

What I am seeing locally:

• Current/day-forward collection works for these feeds.
• Historical coverage starts later than the operating-date window I am trying to backfill for some products.
• Broad historical requests seem to behave as if only a shorter recent request window is searchable/returned.
• For event-based RUC products, some older operating dates return no reports even though the product is active and later dates are available.
• For large hourly state-estimator/dynamic-rating products, the behavior looks different from ordinary public report pagination, so I do not want to assume the same date-range semantics.

Questions:

1. For Secure EWS GetReports / report download flows, should the request StartTime / EndTime be interpreted as report publication/created time rather than operating/delivery time?
2. Are there maximum recommended request-window sizes for these secure report products, especially hourly or high-row-volume products like NP3-217-CD, NP6-627-CD, and NP6-93-CD?
3. Are older secure reports only available through a separate historical records/backfill request process rather than normal EWS date-window pulls?
4. For event-posted products such as NP5-523-CD, NP5-524-CD, and NP5-757-CD, is the correct backfill approach to query around DRUC/HRUC publication windows instead of broad operating-date ranges?
5. If a request window is too broad or outside normal retention, is there an API response field or recommended diagnostic that distinguishes "no reports for this window" from "window too broad / outside searchable history"?
6. Are there product-specific retention limits or publication-window rules documented for these secure products?

I am not including credentials, participant-specific data, or raw secure payloads here. I am only trying to confirm the correct API/request-window semantics so the backfill process can be chunked correctly and avoid misclassifying valid historical gaps as missing data.

Thanks.

[knguyen-ercot]

@iamfaith99 I have communicated your observations to the responsible team at ERCOT. We will investigate. Thank you for your patience.

[iamfaith99]

@knguyen-ercot Don't worry about it, boss. It's long overdue. But thanks for the response. I appreciate it.