Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New libbi action chargefromgrid #16

Merged
merged 21 commits into from
Feb 18, 2024
Merged

New libbi action chargefromgrid #16

merged 21 commits into from
Feb 18, 2024

Conversation

mattburns
Copy link
Contributor

refs issue #15

This change allows you to toggle charging from grid. Toggle on:

python cli.py libbi chargefromgrid true

Disable charging from grid:

python cli.py libbi chargefromgrid false

I won't be offended if you want to re-write my implementation, it just proves the mechanism.

For background:
These actions happen over a different api endpoint that uses AWS Cognito for OAuth authentication.

A simple demonstration of how to interact with this api is like this:

  import requests
  from pycognito import Cognito
  
  u = Cognito('eu-west-2_E57cCJB20','2fup0dhufn5vurmprjkj599041',
      username='youremail')
  
  u.authenticate(password='yourpassword')
  
  headers = {"Authorization": f"Bearer {u.access_token}"}
  print(requests.get("https://myaccount.myenergi.com/api/Product/UserHubsAndDevices", headers=headers).json())
  print(requests.put("https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=true&serialNo=yourserial", headers=headers).json())

@videojedi
Copy link
Contributor

How could the authentication work, if an account is using one of the other methods myenergi support?
I'm using 'sign in with apple' for example. Also google and facebook are offered.

@trizmark
Copy link
Contributor

Once this is merged I have more stuff to add. Got a number of useful endpoints from the Android app...

@FlipGFlop
Copy link

@videojedi - my curl test script uses the serial number of the vhub (my libbi) as the user and the APIKey as the password. This is available from via the 'Advanced' button on the location products page from the web UI, https://myaccount.myenergi.com/location#products.

@mattburns
Copy link
Contributor Author

@FlipGFlop , this project has arguments for username and password which are being used as the serial and apikey.
I assume I'm missing something so rather than refactor something I didn't understand, and introduce a breaking change, this PR adds 2 new arguments: app_email and app_paasword to represent the email auth credentials you login with on the app/website.

Perhaps they should be renamed while keeping the old names as aliases for backwards compatibility?

@videojedi
Copy link
Contributor

Sorry, I'm getting an exception when I try to run....

vscode ➜ / $ myenergi libbi chargefromgrid false
Traceback (most recent call last):
File "/home/vscode/.local/bin/myenergi", line 33, in
sys.exit(load_entry_point('pymyenergi', 'console_scripts', 'myenergi')())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 238, in cli
loop.run_until_complete(main(args))
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 100, in main
await device.set_charge_from_grid(args.arg[0])
File "/home/vscode/pymyenergi/pymyenergi/libbi.py", line 184, in set_charge_from_grid
await self._connection.put(
File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 120, in put
return await self.send("PUT", url, data, oauth)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 75, in send
raise MyenergiException(response.status_code)
pymyenergi.exceptions.MyenergiException

@mattburns
Copy link
Contributor Author

No need to say sorry, will def be my fault, thanks for the feedback!
Do you get the same error if you run it like this?

python cli.py libbi chargefromgrid false

I think you can pass -d for debug logging...

@trizmark
Copy link
Contributor

@FlipGFlop This specific API endpoint uses OAuth, which requires the app username and password. I have just tested it and it does not work with serial number + API key. The serial + API key combo works for API endpoints that use digest auth. Those endpoints are served by sX.myenergi.net.

@trizmark
Copy link
Contributor

@mattburns I am trying to test your fork, but running into issues. I am getting botocore.errorfactory.NotAuthorizedException: An error occurred (NotAuthorizedException) when calling the RespondToAuthChallenge operation: Incorrect username or password.

I have double-checked both and they are correct. I have copied your example cognito test from above and have been successfully using various API endpoints. I will do some more testing/debugging later this afternoon and report back.

@trizmark
Copy link
Contributor

Ok, the auth error was due to me having special characters in my password.

I can confirm that enabling/disabling charging works fine! Excellent work!
If I may have a suggestion: could we change the parameters 'true/false' to 'enable/disable'? They would less 'coder-y' and would be more in line with what you see on the app UI (Enable charging from the grid).

@videojedi
Copy link
Contributor

videojedi commented Sep 28, 2023

tried again, removed special characters from password, but still fails for me.

vscode ➜ ~/pymyenergi (main) $ python cli.py -d libbi chargefromgrid false

DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'PUT']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 400, b'Bad Request', [(b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'92'), (b'Connection', b'keep-alive'), (b'Date', b'Thu, 28 Sep 2023 13:43:58 GMT'), (b'Server', b'Kestrel'), (b'Cache-Control', b'no-cache,no-store'), (b'Expires', b'-1'), (b'Pragma', b'no-cache'), (b'X-Cache', b'Error from cloudfront'), (b'Via', b'1.1 9a9edb00220c3ef50c1919f84fea4888.cloudfront.net (CloudFront)'), (b'X-Amz-Cf-Pop', b'LHR61-P1'), (b'Alt-Svc', b'h3=":443"; ma=86400'), (b'X-Amz-Cf-Id', b'f-2FuEB4ivEcvyJaO4KDcCpa5X1FVq2TjlQkXrW9ntTh1wu7IhD5QA==')])
INFO:httpx:HTTP Request: PUT https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=xxxxxxxx "HTTP/1.1 400 Bad Request"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'PUT']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:pymyenergi.connection:PUT status 400
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
Traceback (most recent call last):
File "/home/vscode/pymyenergi/cli.py", line 4, in
cli()
File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 238, in cli
loop.run_until_complete(main(args))
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 100, in main
await device.set_charge_from_grid(args.arg[0])
File "/home/vscode/pymyenergi/pymyenergi/libbi.py", line 184, in set_charge_from_grid
await self._connection.put(
File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 120, in put
return await self.send("PUT", url, data, oauth)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 75, in send
raise MyenergiException(response.status_code)
pymyenergi.exceptions.MyenergiException

@trizmark
Copy link
Contributor

If it was a username/password issue, then you'd get a different exception. Authentication is done on lines 39-40 of connection.py
self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email) self.oauth.authenticate(password=self.app_password)

Silly question: your debug log shows your S/N as xxxxxxxxx - did you remove it before you posted the log?
Could you post some of the previous lines leading up to the exception?

@videojedi
Copy link
Contributor

Heres the complete log.. (yes I did remove serial number, not sure if thats strictly necessary...)

vscode ➜ ~/pymyenergi (main) $ python cli.py -d libbi chargefromgrid false
DEBUG:pymyenergi.client:Refreshing data for all myenergi devices
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/home/vscode/.local/lib/python3.11/site-packages/certifi/cacert.pem'
DEBUG:pymyenergi.connection:Get Myenergi base url from director
DEBUG:httpcore.connection:connect_tcp.started host='director.myenergi.net' port=443 local_address=None timeout=20 socket_options=None
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98803c10>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026450> server_hostname='director.myenergi.net' timeout=20
DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98663710>
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 401, b'Unauthorized', [(b'Date', b'Thu, 28 Sep 2023 13:43:57 GMT'), (b'Content-Length', b'0'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'WWW-Authenticate', b'Digest realm="MyEnergi Telemetry", nonce="zMY2n0GYLVYVBLcHD5KH0G4zQaGT21d5", opaque="78b66b1db9dc49dd845535e7d7b35287", algorithm=MD5, qop="auth"'), (b'x_myenergi-asn', b'undefined'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net')])
INFO:httpx:HTTP Request: GET https://director.myenergi.net/cgi-jstatus-E "HTTP/1.1 401 Unauthorized"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 28 Sep 2023 13:43:57 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'538'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'x_myenergi-asn', b's18.myenergi.net'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net'), (b'ETag', b'W/"21a-MHybMKcsLiQv/57hZtBM8s4Nocw"')])
INFO:httpx:HTTP Request: GET https://director.myenergi.net/cgi-jstatus-E "HTTP/1.1 200 OK"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
INFO:pymyenergi.connection:Updated myenergi active server to https://s18.myenergi.net
DEBUG:pymyenergi.connection:GET /cgi-get-app-key- https://s18.myenergi.net/cgi-get-app-key-
DEBUG:httpcore.connection:connect_tcp.started host='s18.myenergi.net' port=443 local_address=None timeout=20 socket_options=None
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98603b90>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026450> server_hostname='s18.myenergi.net' timeout=20
DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff98603a50>
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 28 Sep 2023 13:43:57 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'1432'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'x_myenergi-asn', b's18.myenergi.net'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net'), (b'ETag', b'W/"598-ICKKbRcF73eWSkv7kSqGK7wrmhI"')])
INFO:httpx:HTTP Request: GET https://s18.myenergi.net/cgi-get-app-key- "HTTP/1.1 200 OK"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:pymyenergi.connection:GET status 200
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/home/vscode/.local/lib/python3.11/site-packages/certifi/cacert.pem'
DEBUG:pymyenergi.connection:GET /cgi-jstatus-* https://s18.myenergi.net/cgi-jstatus-*
DEBUG:httpcore.connection:connect_tcp.started host='s18.myenergi.net' port=443 local_address=None timeout=20 socket_options=None
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860dad0>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026d50> server_hostname='s18.myenergi.net' timeout=20
DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860da10>
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 200, b'OK', [(b'Date', b'Thu, 28 Sep 2023 13:43:58 GMT'), (b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'2616'), (b'Connection', b'keep-alive'), (b'X-Content-Type-Options', b'nosniff'), (b'X-Frame-Options', b'SAMEORIGIN'), (b'X-XSS-Protection', b'1; mode=block'), (b'Strict-Transport-Security', b'max-age=31536000; includeSubDomains'), (b'x_myenergi-asn', b's18.myenergi.net'), (b'Access-Control-Allow-Credentials', b'true'), (b'Access-Control-Allow-Headers', b'Origin, Content-Type, Accept, Cookie'), (b'Access-Control-Allow-Origin', b'https://admin-ui.s18.myenergi.net'), (b'ETag', b'W/"a38-Z1HoaV0nkKzsnJcGuZ1sr2C8ZXk"')])
INFO:httpx:HTTP Request: GET https://s18.myenergi.net/cgi-jstatus-* "HTTP/1.1 200 OK"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'GET']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:pymyenergi.connection:GET status 200
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
DEBUG:pymyenergi.client:Adding eddi eddi-XXXXXXXX
DEBUG:pymyenergi.client:Adding zappi Zappi Gate
DEBUG:pymyenergi.client:Adding zappi Zappi Door
DEBUG:pymyenergi.client:Adding harvi harvi-XXXXXXXX
DEBUG:pymyenergi.client:Adding libbi libbi-XXXXXXXX
DEBUG:pymyenergi.client:Unknown device type: asn
DEBUG:httpx:load_ssl_context verify=True cert=None trust_env=True http2=False
DEBUG:httpx:load_verify_locations cafile='/home/vscode/.local/lib/python3.11/site-packages/certifi/cacert.pem'
DEBUG:pymyenergi.connection:PUT /api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=XXXXXXXX https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=XXXXXXXX
DEBUG:httpcore.connection:connect_tcp.started host='myaccount.myenergi.com' port=443 local_address=None timeout=20 socket_options=None
DEBUG:httpcore.connection:connect_tcp.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860a110>
DEBUG:httpcore.connection:start_tls.started ssl_context=<ssl.SSLContext object at 0xffff98026600> server_hostname='myaccount.myenergi.com' timeout=20
DEBUG:httpcore.connection:start_tls.complete return_value=<httpcore._backends.anyio.AnyIOStream object at 0xffff9860a050>
DEBUG:httpcore.http11:send_request_headers.started request=<Request [b'PUT']>
DEBUG:httpcore.http11:send_request_headers.complete
DEBUG:httpcore.http11:send_request_body.started request=<Request [b'PUT']>
DEBUG:httpcore.http11:send_request_body.complete
DEBUG:httpcore.http11:receive_response_headers.started request=<Request [b'PUT']>
DEBUG:httpcore.http11:receive_response_headers.complete return_value=(b'HTTP/1.1', 400, b'Bad Request', [(b'Content-Type', b'application/json; charset=utf-8'), (b'Content-Length', b'92'), (b'Connection', b'keep-alive'), (b'Date', b'Thu, 28 Sep 2023 13:43:58 GMT'), (b'Server', b'Kestrel'), (b'Cache-Control', b'no-cache,no-store'), (b'Expires', b'-1'), (b'Pragma', b'no-cache'), (b'X-Cache', b'Error from cloudfront'), (b'Via', b'1.1 9a9edb00220c3ef50c1919f84fea4888.cloudfront.net (CloudFront)'), (b'X-Amz-Cf-Pop', b'LHR61-P1'), (b'Alt-Svc', b'h3=":443"; ma=86400'), (b'X-Amz-Cf-Id', b'f-2FuEB4ivEcvyJaO4KDcCpa5X1FVq2TjlQkXrW9ntTh1wu7IhD5QA==')])
INFO:httpx:HTTP Request: PUT https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?chargeFromGrid=false&serialNo=XXXXXXXX "HTTP/1.1 400 Bad Request"
DEBUG:httpcore.http11:receive_response_body.started request=<Request [b'PUT']>
DEBUG:httpcore.http11:receive_response_body.complete
DEBUG:httpcore.http11:response_closed.started
DEBUG:httpcore.http11:response_closed.complete
DEBUG:pymyenergi.connection:PUT status 400
DEBUG:httpcore.connection:close.started
DEBUG:httpcore.connection:close.complete
Traceback (most recent call last):
File "/home/vscode/pymyenergi/cli.py", line 4, in
cli()
File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 238, in cli
loop.run_until_complete(main(args))
File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/cli.py", line 100, in main
await device.set_charge_from_grid(args.arg[0])
File "/home/vscode/pymyenergi/pymyenergi/libbi.py", line 184, in set_charge_from_grid
await self._connection.put(
File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 120, in put
return await self.send("PUT", url, data, oauth)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/pymyenergi/pymyenergi/connection.py", line 75, in send
raise MyenergiException(response.status_code)
pymyenergi.exceptions.MyenergiException

@trizmark
Copy link
Contributor

This is really puzzling. Can you run the following python code and see what happens? (substitute username_here, password_here and libbi_serialnumber_here)

import requests
from pycognito import Cognito
  
u = Cognito('eu-west-2_E57cCJB20','2fup0dhufn5vurmprjkj599041', username='<username_here>')
  
u.authenticate(password='<password_here>')
  
headers = {"Authorization": f"Bearer {u.access_token}"}
print("Checking if libbi is enabled to charge from grid")
print(requests.get("https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?serialNo=<libbi_serialnumber_here>", headers=headers).json())

This should produce an output like this;

Checking if libbi is enabled to charge from grid
{'status': True, 'message': '', 'field': '', 'content': {'24047164': True}}

The status should be True of False based on whether you have charging from grid enabled or not.

@videojedi
Copy link
Contributor

vscode ➜ ~ $ python test.py
Checking if libbi is enabled to charge from grid
{'status': False, 'message': 'Device not found or user does not have access to it!', 'field': ''}

Ah, OK. Because my account is 'sign in with apple', I registered a new account with email/password and shared my original account with full permissions. I am able to login with the new email/password credentials and control my libbi grid charging from myaccount.myenergi.com......

@trizmark
Copy link
Contributor

So, all OK using pymyenergi as well?

@videojedi
Copy link
Contributor

Sorry, don't understand.

I get the exception listed above from pymyenergi when trying to use the credentials from this 'secondary' account. But, if I log into myaccount.myenergi.com with secondary credentials I am able to see and control all devices.

@trizmark
Copy link
Contributor

Apologies, I didn't understand, but now I do! Let me test this at my end and see if it's workable. I'll set up a secondary account and see if there's a way to make it work.

@trizmark
Copy link
Contributor

Haven't had time to troubleshoot the authentication, but started building already on top of this PR!
Screenshot 2023-09-29 at 17 31 10
That'll be exposed to HA very soon!

@trizmark
Copy link
Contributor

trizmark commented Oct 3, 2023

Just a note that I am running into issues when trying to use Cognito from the HA component due to sync vs async calls.
I am trying to work out what's the best way to deal with this, but feel free to chime in if you have experience in this area.

The error in detail:

2023-10-03 09:32:44.457 WARNING (MainThread) [homeassistant.util.async_] Detected blocking call to putrequest inside the event loop. This is causing stability issues. Please report issue to the custom integration author for myenergi doing blocking calls at custom_components/myenergi/pymyenergi/connection.py, line 39: self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)
2023-10-03 09:32:44.479 ERROR (MainThread) [custom_components.myenergi] Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/botocore/httpsession.py", line 465, in send
    urllib_response = conn.urlopen(
                      ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 714, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 415, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/lib/python3.11/site-packages/botocore/awsrequest.py", line 96, in request
    rval = super().request(method, url, body, headers, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/local/lib/python3.11/http/client.py", line 1286, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/lib/python3.11/http/client.py", line 1297, in _send_request
    self.putrequest(method, url, **skips)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 219, in putrequest
    return _HTTPConnection.putrequest(self, method, url, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 164, in protected_loop_func
    check_loop(func, strict=strict)
  File "/usr/src/homeassistant/homeassistant/util/async_.py", line 151, in check_loop
    raise RuntimeError(
RuntimeError: Blocking calls must be done in the executor or a separate thread; Use `await hass.async_add_executor_job()`; at custom_components/myenergi/pymyenergi/connection.py, line 39: self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)

@mattburns
Copy link
Contributor Author

@trizmark , I'm busy at the mo, but perhaps putting it in the __init__ function was a bad idea. Could you move it into the send function instead? In a lazy-load pattern: If not authed yet then auth etc. ?

@trizmark
Copy link
Contributor

trizmark commented Oct 3, 2023

@mattburns It's actually fine, I have found a way around it. I tried to move things away from __init__, but Cognito itself was not written to be async. HA provides a solution for it though, so it's all good.
I got the charge from grid exposed as a binary sensor at the moment! Working on it to be a proper switch....

@trizmark
Copy link
Contributor

trizmark commented Oct 3, 2023

Ugh, I might need to change stuff around as the OAuth token is only valid for one hour. The implementation in the PR is fine for single calls, but before we can use this in the HA component, I'll need to update things.
Actually, the solution seems to be super-simple. Just need to wait for an hour to see if it really works.

@trizmark
Copy link
Contributor

trizmark commented Oct 3, 2023

Victory! 🎉
image

@trizmark
Copy link
Contributor

trizmark commented Oct 4, 2023

@videojedi To keep the good news rolling: I finally had time to test your special case and you're right, I got the same error. However, there's a solution! After a bit of network sniffing, I figured out what's needed. Can you try the following test with your secondary account?

import requests
from pycognito import Cognito

libbiSerial = ""
appEmail = ""
appPass = ""

u = Cognito('eu-west-2_E57cCJB20','2fup0dhufn5vurmprjkj599041', username=appEmail)
u.authenticate(password=appPass)

headers = {"Authorization": f"Bearer {u.access_token}"}
# grab the list of locations accessible by the user
locs = requests.get("https://myaccount.myenergi.com/api/Locations", headers=headers).json()
# check if guest location - use the first location by default
invId = ''
if locs["content"][0]["isGuestLocation"] == True:
  invId = locs["content"][0]["invitationData"]["invitationId"]
# check if the libbi can charge from the grid
req = "https://myaccount.myenergi.com/api/AccountAccess/LibbiMode?serialNo=" + libbiSerial
if invId != '':
  req = req + "&invitationId=" + invId
d = requests.get(req, headers=headers).json()
chargeFromGrid = d["content"][libbiSerial]

print(f"libbi charge from grid setting: {chargeFromGrid}")

Tech details: with OAuth API endpoints, we need to make a call to api/Locations. From this response we can see if it's a guest location or not. If it's a guest location, we need to include the invitationId in further API calls.

Once you've confirmed that it works for you as well, I can incorporate these changes into pymyenergi.

@videojedi
Copy link
Contributor

@trizmark Brilliant work!

vscode ➜ ~ $ python test.py
libbi charge from grid setting: True

Many thanks.

Richard

@trizmark
Copy link
Contributor

trizmark commented Oct 6, 2023

For those who like to live dangerously...
This is the latest and greatest of the myenergi HA component (plus pymyenergi library). It includes the charge from grid control plus caters for guest account access. Give it a whirl if you want and let me know if it does/doesn't work. I've been running it for a few days, controlling the overnight charging of my libbi. So far, so good!
myenergi-20231006.tgz

@videojedi
Copy link
Contributor

Working for me. Thanks everso!

@trizmark
Copy link
Contributor

@mattburns Do you mind if I PR your fork, so we could get all the libbi charge from grid control related pymyenergy changes into this PR?

@mattburns
Copy link
Contributor Author

@mattburns Do you mind if I PR your fork, so we could get all the libbi charge from grid control related pymyenergy changes into this PR?

Go for it, do whatever you like, I'm a bit busy at the moment to work on this. Thanks

@trizmark
Copy link
Contributor

Go for it, do whatever you like, I'm a bit busy at the moment to work on this. Thanks

OK, PR in. If you merge it, that should update this PR as well, which should set everything up nicely for the HA component update.

@videojedi
Copy link
Contributor

@trizmark Either I've broken it, or sadly the secondary login stuff seems to have stopped working? Even the test code above fails with the following now.

vscode ➜ ~ $ python test.py
Traceback (most recent call last):
File "/home/vscode/.local/lib/python3.11/site-packages/requests/models.py", line 971, in json
return complexjson.loads(self.text, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/init.py", line 346, in loads
return _default_decoder.decode(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/vscode/test.py", line 13, in
locs = requests.get("https://myaccount.myenergi.com/api/Locations", headers=headers).json()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vscode/.local/lib/python3.11/site-packages/requests/models.py", line 975, in json
raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It has been working all week.... something changed on the myenergi server is only possible explanation?

Add extra libbi functionality
@trizmark
Copy link
Contributor

@videojedi it's not just you - it's myenergi

I use the https://myaccount.myenergi.com/api/Locations API endpoint to determine if I need to add the invitation_id to the OAuth calls. This used to return a JSON object with all the accessible locations. Now it returns a 404. 😠

I'll do some network sniffing in the morning to figure out what I need to change.

@trizmark
Copy link
Contributor

The fix was simple. API endpoint has changed from api/Locations to api/Location (became singular for some reason). Updated HA component, which includes the updated pymyenergi library is attached. I have also issued a PR for @mattburns , so we can get this PR up-to-date.
(Hopefully this won't become a regular thing.... 🤞 )

myenergi-20231013.tgz

Updated location discovery API endpoint
@videojedi
Copy link
Contributor

Yep all good and confirmed working again. Many thanks

Copy link
Owner

@CJNE CJNE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution, I'm sorry for the late response!
Before we merge this I would like to make the new authentication optional.
If you don't have a libbi there is no need to for the cognito authentication right?

self.auth = httpx.DigestAuth(self.username, self.password)
self.headers = {"User-Agent": "Wget/1.14 (linux-gnu)"}
self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be optional, check if self.app_email is not empty

self.auth = httpx.DigestAuth(self.username, self.password)
self.headers = {"User-Agent": "Wget/1.14 (linux-gnu)"}
self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)
self.oauth.authenticate(password=self.app_password)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be optional, check I self.app_password if not empty

self.auth = httpx.DigestAuth(self.username, self.password)
self.headers = {"User-Agent": "Wget/1.14 (linux-gnu)"}
self.oauth = Cognito(_USER_POOL_ID, _CLIENT_ID, username=self.app_email)
self.oauth.authenticate(password=self.app_password)
self.oauth_headers = {"Authorization": f"Bearer {self.oauth.access_token}"}
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only if we did the authentication


# Update the extra information available on libbi
# this is the bit that requires OAuth
if existing_device.kind == LIBBI:
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps add a check here to see if app email and password is set and print an error if not?

@trizmark
Copy link
Contributor

trizmark commented Dec 2, 2023

@CJNE No worries. Right now OAuth is only required if you have a libbi. In the future this will change as I saw a note from myenergi that they're slowly trying to migrate from digest to oauth.
Let me make the requested changes.

@trizmark
Copy link
Contributor

@CJNE I've implemented the requested changes. The app_email and app_password parameter is now optional and if they're missing all OAuth stuff is skipped. As part of the libbi only relies on digest, I have left those parts in even if OAuth is missing.
So without OAuth you'll get libbi stats, but you won't be able to see the new things, like the 'charge from grid' and 'charge target'.
Let me know if this is OK or if I should change anything else.

Copy link
Owner

@CJNE CJNE left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, will get this merged :)

@CJNE CJNE merged commit 5966caf into CJNE:main Feb 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants