[Issue] No data update - GrDF web site is half broken - Download button does not work anymore

[ssenart]

Since more 1 week, GrDF web site is half broken and Télécharger button does not work any more.

PyGazpar relies on the same URL behind the Télécharger button. We now have a 404 error on this URL.

GrDF is aware of that issue and is working on it.

Please, be patient !

[pbranly]

Hi
I have the same issue while data are available on the grdf server
I get such error:
https://monespace.grdf.fr/api/e-conso/pce/consommation/informatives/telecharger?dateDebut=2022-10-16&dateFin=2022-12-05&frequence=Journalier&pceList%5B%5D=0112xxxxxxxxx

For your information I also use another système that works

https://github.com/yukulehe/gazpar2mqtt

Any news ?

Regards
Philippe

[ssenart]

Hello, I have a work in progress to move on a new method to get the data.

There is already an alpha release (corresponding to the develop branch) available here: https://pypi.org/project/pygazpar/1.2.0a53/

The API has changed. You will have to adapt you code a little bit. There are some code samples in the README.md.

Let me know

[pbranly]

Ok I will recompile my docker with that version .
And analyse the differences !
Thanks
I will let you know
Phil

[pbranly]

Hi it works
https://github.com/pbranly/pygazpar_to_influxdb

Nevertheless Influx format has changes because I have a lot of errors:

{'time_period': '25/11/2022', 'start_index_m3': 1982, 'end_index_m3': 1986, 'volume_m3': 4, 'energy_kwh': 34, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 7.86, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '26/11/2022', 'start_index_m3': 1986, 'end_index_m3': 1989, 'volume_m3': 4, 'energy_kwh': 34, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 7.16, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '27/11/2022', 'start_index_m3': 1989, 'end_index_m3': 1993, 'volume_m3': 4, 'energy_kwh': 35, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 7.2, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '28/11/2022', 'start_index_m3': 1993, 'end_index_m3': 1996, 'volume_m3': 3, 'energy_kwh': 32, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 7.31, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '29/11/2022', 'start_index_m3': 1996, 'end_index_m3': 2001, 'volume_m3': 5, 'energy_kwh': 45, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 4.09, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '30/11/2022', 'start_index_m3': 2001, 'end_index_m3': 2006, 'volume_m3': 5, 'energy_kwh': 48, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 5.59, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '01/12/2022', 'start_index_m3': 2006, 'end_index_m3': 2011, 'volume_m3': 5, 'energy_kwh': 44, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 4.44, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '02/12/2022', 'start_index_m3': 2011, 'end_index_m3': 2016, 'volume_m3': 5, 'energy_kwh': 51, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 3.15, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

{'time_period': '03/12/2022', 'start_index_m3': 2016, 'end_index_m3': 2024, 'volume_m3': 8, 'energy_kwh': 76, 'converter_factor_kwh/m3': 9.47, 'temperature_degC': 2.25, 'type': 'Mesuré', 'timestamp': '2022-12-05T17:27:57.802315'}

Traceback (most recent call last):

File "/data/./pygazpar_to_influxdb.py", line 90, in

write_api.write(bucket=bucket_influxdb, record=jsonInflux)

File "/usr/local/lib/python3.9/site-packages/influxdb_client/client/write_api.py", line 374, in write

results = list(map(write_payload, payloads.items()))

File "/usr/local/lib/python3.9/site-packages/influxdb_client/client/write_api.py", line 372, in write_payload

return self._post_write(_async_req, bucket, org, final_string, payload[0])

File "/usr/local/lib/python3.9/site-packages/influxdb_client/client/write_api.py", line 483, in _post_write

return self._write_service.post_write(org=org, bucket=bucket, body=body, precision=precision,

File "/usr/local/lib/python3.9/site-packages/influxdb_client/service/write_service.py", line 60, in post_write

(data) = self.post_write_with_http_info(org, bucket, body, **kwargs)  # noqa: E501

File "/usr/local/lib/python3.9/site-packages/influxdb_client/service/write_service.py", line 90, in post_write_with_http_info

return self.api_client.call_api(

File "/usr/local/lib/python3.9/site-packages/influxdb_client/_sync/api_client.py", line 343, in call_api

return self.__call_api(resource_path, method,

File "/usr/local/lib/python3.9/site-packages/influxdb_client/_sync/api_client.py", line 173, in __call_api

response_data = self.request(

File "/usr/local/lib/python3.9/site-packages/influxdb_client/_sync/api_client.py", line 388, in request

return self.rest_client.POST(url,

File "/usr/local/lib/python3.9/site-packages/influxdb_client/_sync/rest.py", line 311, in POST

return self.request("POST", url,

File "/usr/local/lib/python3.9/site-packages/influxdb_client/_sync/rest.py", line 261, in request

raise ApiException(http_resp=r)

influxdb_client.rest.ApiException: (422)

Reason: Unprocessable Entity

HTTP response headers: HTTPHeaderDict({'Content-Type': 'application/json; charset=utf-8', 'X-Influxdb-Build': 'OSS', 'X-Influxdb-Version': 'v2.5.1', 'X-Platform-Error-Code': 'unprocessable entity', 'Date': 'Mon, 05 Dec 2022 16:27:57 GMT', 'Content-Length': '246'})

HTTP response body: {"code":"unprocessable entity","message":"failure writing points to database: partial write: field type conflict: input field "end_index_m3" on measurement "gazpar_consumption_per_day" is type integer, already exists as type float dropped=7"}

some necessary improvment necessary on my side !

Phil

[pbranly]

Hi I try to update my docker as soon as there is a new beta.
Currently I can upload very few datas as grdf site seems « stopped »
Nevertheless I can upload a lot of data’s without too many issues except for the converter factor that seems to change.
Did you notice something around that ?

Thanks
Phil

HTTP response body: {"code":"unprocessable entity","message":"failure writing points to database: partial write: field type conflict: input field "converter_factor_kwh/m3" on measurement "gazpar_consumption_per_day" is type integer, already exists as type float dropped=6"}

[ssenart]

Hi, it complains about the type. It seems to see an int, but it is already used as a float in the db. Don't you have any means to declare the types of your fields.

[pbranly]

Hi
yes I will force the data to be float .
Another question: do you have some kind of release note for the beta evolutions ? For the time being I just change the version number in my docker and recompile but there should be some additional features perhaps or breaking changes ….
Thanks
Phil
Ps: surprisingly I am supposed to receive any modification by mail from you GitHub but I don’t

[ssenart]

Yes, there is an alpha version: see https://github.com/ssenart/PyGazpar/blob/develop/CHANGELOG.md

I'm still currently testing it. The develop branch is here: https://github.com/ssenart/PyGazpar/tree/develop

The last corresponding alpha is 1.2.0a65.

For performance reason, it will be possible to download all frequencies at one time.