Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data for Barbados: generation & demand (fossil, wind, solar, storage charge & discharge) #6459

Closed
q-- opened this issue Feb 6, 2024 · 4 comments · Fixed by #6495
Closed
Labels
help wanted parser buildable! All required data has been found, parser only need to be built

Comments

@q--
Copy link
Contributor

q-- commented Feb 6, 2024

Production

Hourly solar/wind/fossil/discharge graphs for ~1½ day ago (date at the bottom of each graph): https://dataservices.acelerex.com/barbados/grid-operation/generation/current

  • At the top of the page it says "Values: <date> <time>", but the values at the top of the page do not align with the values in the graphs at the same time stamp. I wondered if it could be (near) real-time generation, but it turned out that if I converted that timestamp from my time zone to Barbados time I found a matching data point in the graph at that time.

Consumption

Hourly demand and storage charging graphs for ~1½ day ago: https://dataservices.acelerex.com/barbados/grid-operation/demand/current

  • At the top of the page it says "Values:", this time without a date or timestamp. Again there was a matching data point if I converted the last whole hour local time to Barbados time.

Note: I found the dashboard embedded at this page

@VIKTORVAV99 VIKTORVAV99 added the parser buildable! All required data has been found, parser only need to be built label Feb 6, 2024
@VIKTORVAV99
Copy link
Member

@q-- should I label this as help wanted or do you plan on doing it yourself?

@q--
Copy link
Contributor Author

q-- commented Feb 6, 2024

@q-- should I label this as help wanted or do you plan on doing it yourself?

Feel free to label this and the other "parser buildable" issues as "help wanted".

(Except #5426 and #5552, which both have had an associated pull request for several months which could go live if it passes review)

@VIKTORVAV99
Copy link
Member

VIKTORVAV99 commented Feb 6, 2024

I'll go through the other issues tomorrow and make sure the PRs are reviewed and moved forward.

@Ugzuzg
Copy link
Contributor

Ugzuzg commented Feb 17, 2024

The data is fetched by calling an API endpoint: https://dataservices.acelerex.com/api/barbados/uploads?name=operational-data&$sort[created_at]=-1. The endpoint gives access to a "barbados"."files" PostgreSQL table. Another available file type is connections-data (it contains [ "geometry", "LATITUDE", "LONGITUDE", "Feeder ID", "Meter ID", "Parish", "GRIDNUMBER", "TCAP", "EFFECTIVE DATE", "" ]), but operational-data is what we need.

operational-data has information in the following format:

                  datetime 24 hr monitored solar production 24 hr monitored wind production 24 hr storage charging 24 hr storage discharging 24 hr fossil production 24 hr Net demand 24 hr Curtailment
0   2024-02-16T00:00-04:00                                0                               0                   -2.2                         0                   116.3            114.1                 0
1   2024-02-16T01:00-04:00                                0                               0                   -2.3                         0                     112            109.7                 0
2   2024-02-16T02:00-04:00                                0                               0                     -1                         0                   108.5            107.5                 0
3   2024-02-16T03:00-04:00                                0                               0                      0                         0                   105.6            105.6                 0
4   2024-02-16T04:00-04:00                                0                               0                      0                         0                     104              104                 0
5   2024-02-16T05:00-04:00                                0                               0                      0                         0                     109              109                 0
6   2024-02-16T06:00-04:00                                0                               0                      0                         0                   114.7            114.7                 0
7   2024-02-16T07:00-04:00                              0.1                               0                      0                         0                   118.1            118.2                 0
8   2024-02-16T08:00-04:00                              2.6                               0                      0                         0                   112.6            115.2                 0
9   2024-02-16T09:00-04:00                                5                               0                      0                         0                   100.8            105.8                 0
10  2024-02-16T10:00-04:00                                5                               0                      0                         0                    98.9            103.9                 0
11  2024-02-16T11:00-04:00                                7                               0                      0                         0                    94.5            101.5                 0
12  2024-02-16T12:00-04:00                                6                               0                      0                         0                    86.1             92.1                 0
13  2024-02-16T13:00-04:00                                7                               0                      0                         0                    85.6             92.6                 0
14  2024-02-16T14:00-04:00                                7                               0                      0                         0                    95.7            102.7                 0
15  2024-02-16T15:00-04:00                              4.9                               0                      0                         0                   100.9            105.8                 0
16  2024-02-16T16:00-04:00                              1.3                               0                      0                         0                   116.4            117.7                 0
17  2024-02-16T17:00-04:00                              2.1                               0                      0                         0                   127.2            129.3                 0
18  2024-02-16T18:00-04:00                                0                               0                      0                       2.6                   134.3            136.9                 0
19  2024-02-16T19:00-04:00                                0                               0                      0                       2.5                   139.6            142.1                 0
20  2024-02-16T20:00-04:00                                0                               0                      0                       2.4                   134.9            137.3                 0
21  2024-02-16T21:00-04:00                                0                               0                      0                         0                   133.3            133.3                 0
22  2024-02-16T22:00-04:00                                0                               0                      0                         0                     126              126                 0
23  2024-02-16T23:00-04:00                                0                               0                      0                         0                   120.1            120.1                 0

I've guessed some useful query params:

  • $limit changes the default number of records (days) fetched from 1 to a different number.
  • created_at[$lt]=2024-02-10 filters documents by the date, allowing to fetch historical data.

By reverting the sort order, I can see that the data has started being created on 2022-11-30. There are some differences to the output format, and it looks like they were still fixing some bugs in the first days, as the responses contain "[object Object]" for daily values.

Properly timestamped datapoints start appearing on 2023-02-28 with some gaps between days.


I wouldn't have to guess if I'd follow the link without authenticating. It says right there that it uses feathers.js, so here are all supported query params: https://feathersjs.com/api/databases/querying.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted parser buildable! All required data has been found, parser only need to be built
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants