How to test new implementations? #30

rct · 2023-08-07T20:36:33Z

rct
Aug 7, 2023

Very glad to see #13, the implementation for ConEd is already merged.

What's the best way to test it and try it out?

Thanks!

tronikos · 2023-08-08T06:08:59Z

tronikos
Aug 8, 2023
Maintainer

wait for Add support for MFA auth in opower home-assistant/core#97878 to be merged and released, likely in 2023.9
manually apply the changes of that PR to your local installation, depending on your installation this is hard for someone not familiar with docker commands
create a directory /config/custom_components/opower and add all files from the core integration, including the modified files from that PR. You will also have to add version in the manifest.

0 replies

rct · 2023-08-08T15:29:55Z

rct
Aug 8, 2023
Author

@tronikos - Thanks for your response. I will try option 3, making a custom_component on a test hass instance. I might also try 2. (I had also found your testing HACS repo that now says it is out of date.

Side Note: the reasons I'd like to test this out before waiting for 2023.9 is that I have multiple ConEd accounts under the same username. Is that something that is supported by any chance?

testing with demo.py

After taking a 2nd look at the opower repo, I decided to try to see if I could get demo.py to work.

Results/exceptions so far -- I think I have to change my ConEd auth methods. Currently it is set to send via SMS an OTP on login. Where would a user find out the utility specific account auth requirements?

Just following demo.py's prompts and entering username and password:

Traceback (most recent call last):
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/demo.py", line 151, in <module>
    asyncio.run(_main())
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/demo.py", line 77, in _main
    await opower.async_login()
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/opower/opower.py", line 193, in async_login
    self.access_token = await self.utility.async_login(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/opower/utilities/coned.py", line 80, in async_login
    raise InvalidAuth("TOTP secret is required for MFA accounts")
opower.exceptions.InvalidAuth: TOTP secret is required for MFA accounts

Trying to supply the OTP password as the mfa_secret -- `binascii.Error: Incorrect padding`

Tried:
`python src/demo.py --utility coned --username --mfa_secret <most recent 6 digit OTP from SMS>

Traceback (most recent call last):
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/demo.py", line 151, in <module>
    asyncio.run(_main())
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/demo.py", line 77, in _main
    await opower.async_login()
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/opower/opower.py", line 193, in async_login
    self.access_token = await self.utility.async_login(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/opower/utilities/coned.py", line 83, in async_login
    mfaCode = TOTP(optional_mfa_secret).now()
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pyvenvs/opower/lib/python3.11/site-packages/pyotp/totp.py", line 64, in now
    return self.generate_otp(self.timecode(datetime.datetime.now()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/pyvenvs/opower/lib/python3.11/site-packages/pyotp/otp.py", line 35, in generate_otp
    hasher = hmac.new(self.byte_secret(), self.int_to_bytestring(input), self.digest)
                      ^^^^^^^^^^^^^^^^^^
  File "/srv/pyvenvs/opower/lib/python3.11/site-packages/pyotp/otp.py", line 52, in byte_secret
    return base64.b32decode(secret, casefold=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/base64.py", line 254, in b32decode
    return _b32decode(_b32alphabet, s, casefold, map01)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/base64.py", line 240, in _b32decode
    raise binascii.Error('Incorrect padding')
binascii.Error: Incorrect padding

0 replies

rct · 2023-08-08T15:36:13Z

rct
Aug 8, 2023
Author

So I'm guessing the ConEd MFA (aka Two step verification) needs to be "Google Authenticator" for opower, and I have to capture the TOTP secret so that opower can generate the necessary TOTP.

When I tried that, the ConEd website said " Error: Please call us at 1-800-752-6633 to complete this request. "

Before I call them and try to deal with that, I''ll wait for your confirmation on how the account should be set up.

0 replies

tronikos · 2023-08-08T18:19:09Z

tronikos
Aug 8, 2023
Maintainer

The ConEd contributor hasn't updated the documentation yet. Based on the discussion in the PR and the code, my understanding is that only the TOTP secret is implemented. https://github.com/bvlaicu/coned seems to have some relevant instructions:

For MFA Security Question, to set up your MFA secret (answer), log into coned.com (or oru.com), go to your profile and reset your 2FA method. When setting up 2FA again, there will be option to say you do not have texting on your phone. Select this and you should be able to use a security question instead.

For MFA TOTP, choose Google Authenticator, choose a device type and when presented with the QR code, click on "Can't scan?". It should provide you with the MFA secret.

0 replies

rct · 2023-08-08T21:34:38Z

rct
Aug 8, 2023
Author

@tronikos - Thanks for your help. After about 20+ minutes on the phone with ConEd, I was able to get them to resolve the error so I could switch my account over to using Google Auth TOTP.

The demo script does work and returns data for the active electric meter on my 2nd account. (The 1st account has gas and a disabled electric meter.) So I'm glad to see the demo works and now time to try getting this working with Home Assistant.

I get the exception from demo about a mismatched aggregation when it tries to fetch billing data vs. usage data with either the default day or hour. I didn't see a CLI arg to turn that off fetching billing data, but that's certainly not important for a demo script.

Getting historical data: account= Account(customer=Customer(uuid='xxx-xxx'), uuid='xxx-xxx', utility_account_id='xxx', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.BILLING: 'BILLING'>) aggregate_type= hour start_date= 2023-08-01 17:24:13.543517 end_date= 2023-08-08 17:24:13.543581
Traceback (most recent call last):
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/demo.py", line 151, in <module>
    asyncio.run(_main())
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/anaconda/anaconda3/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/demo.py", line 121, in _main
    cost_data = await opower.async_get_cost_reads(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/opower/opower.py", line 318, in async_get_cost_reads
    reads = await self._async_get_dated_data(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/s2/home/rct/proj/coned/tronikos/opower/src/opower/opower.py", line 389, in _async_get_dated_data
    raise ValueError(
ValueError: Requested aggregate_type: hour not supported by account's read_resolution: BILLING

0 replies

tronikos · 2023-08-08T21:57:01Z

tronikos
Aug 8, 2023
Maintainer

There is --aggregate_type bill which might not work under Linux, see #28. Instead you can just change the default value in the code.

2 replies

rct Aug 8, 2023
Author

I should have mentioned, --aggregate_type bill does work for me and gets both gas and electric bills across the two accounts. I'm running on Linux, though I'm using Python 3.11 from Anaconda at the moment, not seeing the issue in #28.

tronikos Aug 8, 2023
Maintainer

The demo is using the same aggregate_type for all accounts so it's expected to encounter that exception. I guess we can change the demo to use a supported aggregate type to avoid it.

rct · 2023-08-09T15:54:59Z

rct
Aug 9, 2023
Author

Trying to test the updated integration with Home Assistant. Right now I'm going the custom component route. When I try to configure up the integration, I get:

Config entry 'conEdison (ConEd) (<emailaddr>)' for opower integration could not authenticate: 2FA code was invalid. Is the secret wrong?

I may be missing something. (I've pulled in the updated components from the two PRs and merged the changes to coordinator.py from both. I'll go back over my steps.

Things I've noticed:

ConEd doesn't sort correctly in the list of utilities, but is there.
The TOTP secret field in the config dialog doesn't have a label
There is no way to reconfigure the opower integration to change password/totp secret without deleting and readding the integration.

9 replies

Sebmaster Aug 9, 2023

I wouldn't recommend frankenstein-ing the plugins together like that. You should be able to just copy the whole opower folder from the homeassistant-core commit into your custom_components folder (+ add version key to manifest). The -hacs repo shouldn't be involved.

I just rebased the HA PR and tested on my side so no other changes should be necessary.

rct Aug 9, 2023
Author

@Sebmaster thanks for your help. I still get the same error. I have:

removed the custom_components/opower with the 2 PRs merged
Created a new custom_components/opower using homeassistant/core:dev + pulling in Add support for MFA auth in opower home-assistant/core#97878
Added a manifest.json that depends on opower 0.0.24 and has a version key.
Deleted the old opower integration
Restart home assistant

When I add the integration, after a number of seconds, I get a "Success" pop up after a number of seconds, but the integration says "failed"

I do get (and was getting the dialog about the integration needing attention, where I get prompted again for password and the TOTP secret (without the label).

After re-entering it, I still get a "needs attention" and the opower integration is shown with no devices/entities.

This is what I have in my logs:

2023-08-09 13:28:02.470 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration opower which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant

...

2023-08-09 13:30:22.039 DEBUG (MainThread) [custom_components.opower.coordinator] Finished fetching Opower data in 2.168 seconds (success: False)
2023-08-09 13:30:22.039 WARNING (MainThread) [homeassistant.config_entries] Config entry 'conEdison (ConEd) (<email>)' for opower integration could not authenticate: 2FA code was invalid. Is the secret wrong?
2023-08-09 13:38:40.027 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []
2023-08-09 13:38:40.033 DEBUG (MainThread) [custom_components.opower.coordinator] Finished fetching Opower data in 6.590 seconds (success: True)

rct Aug 9, 2023
Author

If I look at .storage/core.config_entries, the data section under domain opower has the right username, password, and totp secret that all work with the demo.py from the opower repo.

As you can see I have debug logging enabled, but I'm not getting any other clues from the log.

Also, I'm running this custom component against the released hass 2023.8.1.

Sebmaster Aug 9, 2023

Added a manifest.json that depends on opower 0.0.24 and has a version key.

If you're correctly based on the PR, the dependency change should not be necessary the manifest should already exist and only need the version property, no other changes.

rct Aug 9, 2023
Author

Yes, you are correct, though it doesn't result in a difference.

diff /tmp/opower-git/manifest.json manifest.json
9c9,10
<   "requirements": ["opower==0.0.24"]
---
>   "requirements": ["opower==0.0.24"],
>   "version": "0.0.0"

Here's my custom component:

➜  opower pwd; ls -l
/root/config/custom_components/opower
total 48
-rw-rw-r--    1 hassio   hassio         989 Aug  9 13:06 __init__.py
drwxr-xr-x    2 root     root          4096 Aug  9 13:24 __pycache__
-rw-rw-r--    1 hassio   hassio        4068 Aug  9 13:12 config_flow.py
-rw-rw-r--    1 hassio   hassio         122 Aug  9 13:12 const.py
-rw-rw-r--    1 hassio   hassio        8618 Aug  9 13:12 coordinator.py
-rwxr-xr-x    1 root     root           295 Aug  9 13:18 manifest.json
-rw-rw-r--    1 hassio   hassio        8645 Aug  9 13:06 sensor.py
-rw-rw-r--    1 hassio   hassio        1073 Aug  9 13:12 strings.json

➜  opower sha256sum *py *json
76240ac3aee38b75229162cdfde186b69ca30ddadcee986fefbc77b802067622  __init__.py
62379efe1e16c45615eb90232020e17d117ac7b9cb462fe9c0e8540439a2b018  config_flow.py
2f13c2c99ef97a7dd3efdd2f03d87a214df7cd74c2098884a93eb8720b8e04a6  const.py
7bbbaeb3073b8e3aac16a1b925337140acfec3a76057f5e1f5a791da3612d68f  coordinator.py
eddfbccb102c7512e823359ed430f7a616a16840de520066412621d102fcfa32  sensor.py
f2914c76a50e98701ec21aa46cecf4a1fde5b776e46eff7d0c01b9f1f0b08a06  manifest.json
25ae70e4fb0e0c5d48fe3aa3125a579408ad848ff022aa751c22f9ba26a1710c  strings.json

➜  opower-git sha256sum *py *json
76240ac3aee38b75229162cdfde186b69ca30ddadcee986fefbc77b802067622  __init__.py
62379efe1e16c45615eb90232020e17d117ac7b9cb462fe9c0e8540439a2b018  config_flow.py
2f13c2c99ef97a7dd3efdd2f03d87a214df7cd74c2098884a93eb8720b8e04a6  const.py
7bbbaeb3073b8e3aac16a1b925337140acfec3a76057f5e1f5a791da3612d68f  coordinator.py
eddfbccb102c7512e823359ed430f7a616a16840de520066412621d102fcfa32  sensor.py
ef228879b40fdb641093aadca535a82df2ebb946b0652ae9449b4761660c455c  manifest.json
25ae70e4fb0e0c5d48fe3aa3125a579408ad848ff022aa751c22f9ba26a1710c  strings.json

$ git show
commit dde75f43371c798190b4ed9e097a8689679ebd32 (HEAD -> feature/opower-mfa)
Author: Sebastian Mayr <git@smayr.name>
Date:   Sat Aug 5 20:52:23 2023 -0400

    Add support for MFA auth in opower

diff --git a/homeassistant/components/opower/config_flow.py b/homeassistant/components/opower/config_flow.py
index fdf007c3b6..d2273c605d 100644
--- a/homeassistant/components/opower/config_flow.py
+++ b/homeassistant/components/opower/config_flow.py
@@ -14,15 +14,18 @@ from homeassistant.core import HomeAssistant
 from homeassistant.data_entry_flow import FlowResult
 from homeassistant.helpers.aiohttp_client import async_create_clientsession

-from .const import CONF_UTILITY, DOMAIN
+from .const import CONF_TOTP_SECRET, CONF_UTILITY, DOMAIN

 _LOGGER = logging.getLogger(__name__)

 STEP_USER_DATA_SCHEMA = vol.Schema(
     {
-        vol.Required(CONF_UTILITY): vol.In(get_supported_utility_names()),
+        vol.Required(CONF_UTILITY): vol.In(
+            get_supported_utility_names(supports_mfa=True)
+        ),
         vol.Required(CONF_USERNAME): str,
         vol.Required(CONF_PASSWORD): str,
+        vol.Optional(CONF_TOTP_SECRET): str,
     }
 )

@@ -36,6 +39,7 @@ async def _validate_login(
         login_data[CONF_UTILITY],
         login_data[CONF_USERNAME],
         login_data[CONF_PASSWORD],
+        login_data.get(CONF_TOTP_SECRET, None),
     )
     errors: dict[str, str] = {}
     try:
@@ -106,6 +110,7 @@ class OpowerConfigFlow(config_entries.ConfigFlow, domain=DOMAIN):
                 {
                     vol.Required(CONF_USERNAME): self.reauth_entry.data[CONF_USERNAME],
                     vol.Required(CONF_PASSWORD): str,
+                    vol.Optional(CONF_TOTP_SECRET): str,
                 }
             ),
             errors=errors,
diff --git a/homeassistant/components/opower/const.py b/homeassistant/components/opower/const.py
index b996a214a0..c07d41bbdc 100644
--- a/homeassistant/components/opower/const.py
+++ b/homeassistant/components/opower/const.py
@@ -3,3 +3,4 @@
 DOMAIN = "opower"

 CONF_UTILITY = "utility"
+CONF_TOTP_SECRET = "totp_secret"
diff --git a/homeassistant/components/opower/coordinator.py b/homeassistant/components/opower/coordinator.py
index c331f45bc4..bb30c9b16d 100644
--- a/homeassistant/components/opower/coordinator.py
+++ b/homeassistant/components/opower/coordinator.py
@@ -27,7 +27,7 @@ from homeassistant.exceptions import ConfigEntryAuthFailed
 from homeassistant.helpers import aiohttp_client
 from homeassistant.helpers.update_coordinator import DataUpdateCoordinator


-from .const import CONF_UTILITY, DOMAIN
+from .const import CONF_TOTP_SECRET, CONF_UTILITY, DOMAIN

 _LOGGER = logging.getLogger(__name__)

@@ -54,6 +54,7 @@ class OpowerCoordinator(DataUpdateCoordinator[dict[str, Forecast]]):
             entry_data[CONF_UTILITY],
             entry_data[CONF_USERNAME],
             entry_data[CONF_PASSWORD],
+            entry_data.get(CONF_TOTP_SECRET, None),
         )

     async def _async_update_data(
diff --git a/homeassistant/components/opower/strings.json b/homeassistant/components/opower/strings.json
index 037983eb6f..ac931bf930 100644
--- a/homeassistant/components/opower/strings.json
+++ b/homeassistant/components/opower/strings.json
@@ -5,14 +5,16 @@
         "data": {
           "utility": "Utility name",
           "username": "[%key:common::config_flow::data::username%]",
-          "password": "[%key:common::config_flow::data::password%]"
+          "password": "[%key:common::config_flow::data::password%]",
+          "totp_secret": "TOTP Secret (only for some utilities, see documentation)"
         }
       },
       "reauth_confirm": {
         "title": "[%key:common::config_flow::title::reauth%]",
         "data": {
           "username": "[%key:common::config_flow::data::username%]",
-          "password": "[%key:common::config_flow::data::password%]"
+          "password": "[%key:common::config_flow::data::password%]",
+          "totp_secret": "TOTP Secret (only for some utilities, see documentation)"
         }
       }
     },

[omitted changes to test_config_flow.py]

I get that it works on your machine and you think this is pilot error on my part. If you think there are other things I should be doing/showing, please let me know.

How can I debug this? Debug logging for the integration isn't given me much. I don't think I'm getting debugging messages from the opower module itself.

rct · 2023-08-09T18:06:44Z

rct
Aug 9, 2023
Author

@tronikos mentioned this in the PR

Regarding reconfiguration: I think that may be a regular bug in the integration and unrelated to the TOTP change? All auth issues that are being encountered should just trigger the regular reauth flow (admittedly I've never seen it, but I also haven't tried to trigger it).

The re-auth dialog (aka integration needs attention) does pop-up, and I've tried it with the same results, but it also seems to go away. I think it goes away after the update message declaring success happens.

2023-08-09 14:01:40.437 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []

0 replies

Sebmaster · 2023-08-09T20:15:30Z

Sebmaster
Aug 9, 2023

Just read up a bit in the thread, was only responding to individual messages before.

Since the demo works for you it sounds like this has something to do with the device token - possibly specifically the previously identified issue of TOTP token reuse. I'm theorizing that your HA instance is trying to call authenticate multiple times in quick succession (presumably once for the login dialog, then again for the sync).

To get into this state would require two things: 1. your device token cookie is (not) being set/reset/not accepted by ConEd, or 2. HomeAssistant is resetting the cookie/not reusing the same session on its side. This will cause ConEd to re-prompt for 2FA and since its in such quick succession, you'll reuse the same TOTP code and ConEd will reject the second auth attempt. The demo script does the double auth "properly" (tries authenticating twice, reusing the device token once).

So now, to resolve this: as an initial workaround, try just reloading the integration. This should trigger the sync without the initial login, thereby only causing one auth flow where the TOTP code should be accepted.

What version of HA are you using?

17 replies

rct Aug 10, 2023
Author

Regarding forecasts for ConEd, I modified a opower so I could see some activity from home assistant. It looks to me like some forecast data is returned from one API call combined-forecast to one account, but not the other. Unfortunately the one that works is called first, so possibly the one that failed is overwriting and/or breaking the control flow?

2023-08-09 16:36:52.527 DEBUG (MainThread) [/config/opower/opower.py] Fetching: https://cned.opower.com/ei/edge/apis/bill-forecast-cws-v1/cws/cned/customers/xxxx/combined-forecast
2023-08-09 16:36:53.059 DEBUG (MainThread) [/config/opower/opower.py] Fetched: {
  "isValidUser": true,
  "totalForecast": {
    "meterType": "COMBINED",
    "startDate": "2023-07-12",
    "endDate": "2023-08-10",
    "currentDate": "2023-08-09",
    "daysInPeriod": 30,
    "currentDay": 29,
    "daysLeftInBill": 1,
    "forecastedUsage": 753,
    "forecastedCost": 0,
    "typicalUsage": 1227,
    "typicalCost": 0,
    "budgetBilling": false,
    "costToDate": 0,
    "usageToDate": 701,
    "currencySymbol": "$"
  },
  "totalMetadata": [
    "NO_FORECASTED_COST"
  ],
  "accountForecasts": [
    {
      "unitOfMeasure": "KWH",
      "meterType": "ELEC",
      "startDate": "2023-07-12",
      "endDate": "2023-08-10",
      "currentDate": "2023-08-09",
      "daysInPeriod": 30,
      "currentDay": 29,
      "daysLeftInBill": 1,
      "forecastedUsage": 753,
      "typicalUsage": 1227,
      "budgetBilling": false,
      "usageToDate": 701,
      "currencySymbol": "$",
      "preferredUtilityAccountId": "nnnnnnn",
      "accountUuids": [
        "xxxx-xxxx"
      ],
      "isSolar": false
    }
  ]
}
2023-08-09 16:36:53.060 DEBUG (MainThread) [/config/opower/opower.py] Fetching: https://cned.opower.com/ei/edge/apis/bill-forecast-cws-v1/cws/cned/customers/xxxxxxx/combined-forecast
2023-08-09 16:36:53.188 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []

(This was from yesterday afternoon, there haven't been any other fetches since then.)

rct Aug 11, 2023
Author

@tronikos - based on the forecast response I posted above, should I be getting sensors for ConEd -- at least if there wasn't issues/confusion with multiple accounts?

@Sebmaster - Has anyone with a more standard ConEd account (1 electric meter and 1 gas meter) tested things?

Also, I've found that my results aren't reliable/predictable. After getting things working on the test Hass VM on my laptop yesterday, I tried to move the configuration over to my actual "production" hass instance, so I'd get data while still in peak air conditioning season.

When adding the integration, this time I did NOT get the 2FA error, possibly due to timing as @Sebmaster mentioned.

However, there are no errors in the logs and no statistics were created. I've checked from both the energy dashboard and developer tools, no statistics entries.

I see this in the logs giving me the indication that the integration is trying to do something.

2023-08-10 15:55:59.536 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []
2023-08-10 15:55:59.536 DEBUG (MainThread) [custom_components.opower.coordinator] Finished fetching Opower data in 21.646 seconds (success: True)
2023-08-11 08:43:50.736 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []
2023-08-11 08:43:50.736 DEBUG (MainThread) [custom_components.opower.coordinator] Finished fetching Opower data in 7.969 seconds (success: True)

I think I should probably move some of this into issues against opower/coned. I have no idea how much of this is related to my account with 3 meters.

tronikos Aug 12, 2023
Maintainer

Can you rebase? I had some fixes upstream. Specifically home-assistant/core#98144 should likely fix your issue. Without it, it's expected to not get any statistics when you don't get forecasts. In your case it is unexpected you don't get any forecasts though. I don't remember if you said that earlier, do you get forecasts when you run the demo.py?

rct Aug 12, 2023
Author

@tronikos wrote:

Can you rebase? I had some fixes upstream. Specifically home-assistant/core#98144 should likely fix your issue.

Ok, I'll give it a shot. I still need to merge the changes home-assistant/core#97878 to get MFA auth for ConEd. -- coordinator.py has a bunch of updates. That PR is still in draft and may need merging/rebasing given the other changes.

I don't remember if you said that earlier, do you get forecasts when you run the demo.py?

I don't get forecasts from demo.py because it dies from the exception. However based on the debugging output I posted above, it does look like forecast data is being returned from one account, but not from another account which I think is causing it not to be processed.

Here's the output reposted:

2023-08-09 16:36:52.527 DEBUG (MainThread) [/config/opower/opower.py] Fetching: https://cned.opower.com/ei/edge/apis/bill-forecast-cws-v1/cws/cned/customers/xxxx/combined-forecast
2023-08-09 16:36:53.059 DEBUG (MainThread) [/config/opower/opower.py] Fetched: {
  "isValidUser": true,
  "totalForecast": {
    "meterType": "COMBINED",
    "startDate": "2023-07-12",
    "endDate": "2023-08-10",
    "currentDate": "2023-08-09",
    "daysInPeriod": 30,
    "currentDay": 29,
    "daysLeftInBill": 1,
    "forecastedUsage": 753,
    "forecastedCost": 0,
    "typicalUsage": 1227,
    "typicalCost": 0,
    "budgetBilling": false,
    "costToDate": 0,
    "usageToDate": 701,
    "currencySymbol": "$"
  },
  "totalMetadata": [
    "NO_FORECASTED_COST"
  ],
  "accountForecasts": [
    {
      "unitOfMeasure": "KWH",
      "meterType": "ELEC",
      "startDate": "2023-07-12",
      "endDate": "2023-08-10",
      "currentDate": "2023-08-09",
      "daysInPeriod": 30,
      "currentDay": 29,
      "daysLeftInBill": 1,
      "forecastedUsage": 753,
      "typicalUsage": 1227,
      "budgetBilling": false,
      "usageToDate": 701,
      "currencySymbol": "$",
      "preferredUtilityAccountId": "nnnnnnn",
      "accountUuids": [
        "xxxx-xxxx"
      ],
      "isSolar": false
    }
  ]
}
2023-08-09 16:36:53.060 DEBUG (MainThread) [/config/opower/opower.py] Fetching: https://cned.opower.com/ei/edge/apis/bill-forecast-cws-v1/cws/cned/customers/xxxxxxx/combined-forecast
2023-08-09 16:36:53.188 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []

rct Aug 12, 2023
Author

@tronikos - results from Pulling in opower from homeassistant-dev and merging in the MFA changes. Data is being pulled. No sensors still.

2023-08-12 15:36:20.752 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []
2023-08-12 15:36:32.071 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []
2023-08-12 15:36:48.760 DEBUG (MainThread) [custom_components.opower.coordinator] Updating sensor data with: []

Now I get the same exceptions I get with demo.py:

ValueError: Requested aggregate_type: hour not supported by account's read_resolution: BILLING

I get two of those exceptions for the two accounts that have:

"readResolution": "BILLING"

2023-08-12 15:36:49.331 ERROR (MainThread) [custom_components.opower.coordinator] Unexpected error fetching Opower data: Requested aggregate_type: hour not supported by account's read_resolution: BILLING
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 283, in _async_refresh
    self.data = await self._async_update_data()
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opower/coordinator.py", line 73, in _async_update_data
    await self._insert_statistics()
  File "/config/custom_components/opower/coordinator.py", line 104, in _insert_statistics
    cost_reads = await self._async_get_recent_cost_reads(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/opower/coordinator.py", line 214, in _async_get_recent_cost_reads
    return await self.api.async_get_cost_reads(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/opower/opower.py", line 332, in async_get_cost_reads
    reads = await self._async_get_dated_data(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/opower/opower.py", line 403, in _async_get_dated_data
    raise ValueError(
ValueError: Requested aggregate_type: hour not supported by account's read_resolution: BILLING

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to test new implementations? #30

{{title}}

Replies: 9 comments 28 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

How to test new implementations? #30

rct Aug 7, 2023

Replies: 9 comments · 28 replies

tronikos Aug 8, 2023 Maintainer

rct Aug 8, 2023 Author

testing with demo.py

Just following demo.py's prompts and entering username and password:

Trying to supply the OTP password as the mfa_secret -- binascii.Error: Incorrect padding

rct Aug 8, 2023 Author

tronikos Aug 8, 2023 Maintainer

rct Aug 8, 2023 Author

tronikos Aug 8, 2023 Maintainer

rct Aug 8, 2023 Author

tronikos Aug 8, 2023 Maintainer

rct Aug 9, 2023 Author

Sebmaster Aug 9, 2023

rct Aug 9, 2023 Author

rct Aug 9, 2023 Author

Sebmaster Aug 9, 2023

rct Aug 9, 2023 Author

rct Aug 9, 2023 Author

Sebmaster Aug 9, 2023

rct Aug 10, 2023 Author

rct Aug 11, 2023 Author

tronikos Aug 12, 2023 Maintainer

rct Aug 12, 2023 Author

rct Aug 12, 2023 Author

rct
Aug 7, 2023

Replies: 9 comments 28 replies

tronikos
Aug 8, 2023
Maintainer

rct
Aug 8, 2023
Author

Trying to supply the OTP password as the mfa_secret -- `binascii.Error: Incorrect padding`

rct
Aug 8, 2023
Author

tronikos
Aug 8, 2023
Maintainer

rct
Aug 8, 2023
Author

tronikos
Aug 8, 2023
Maintainer

rct Aug 8, 2023
Author

tronikos Aug 8, 2023
Maintainer

rct
Aug 9, 2023
Author

rct Aug 9, 2023
Author

rct Aug 9, 2023
Author

rct Aug 9, 2023
Author

rct
Aug 9, 2023
Author

Sebmaster
Aug 9, 2023

rct Aug 10, 2023
Author

rct Aug 11, 2023
Author

tronikos Aug 12, 2023
Maintainer

rct Aug 12, 2023
Author

rct Aug 12, 2023
Author