Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smart Meter Texas (SMT) Unable to connect with Integration #101166

Closed
james-wiederkehr opened this issue Sep 30, 2023 · 45 comments · Fixed by #116321
Closed

Smart Meter Texas (SMT) Unable to connect with Integration #101166

james-wiederkehr opened this issue Sep 30, 2023 · 45 comments · Fixed by #116321

Comments

@james-wiederkehr
Copy link

The problem

HA's integration with SMT is unable to poll the sensor. I removed and tried to add the integration, it says it can't connect. Unfortunately, since I can't get it setup again after removing it, I don't have the diagnostic file and didn't download before removing it.

What version of Home Assistant Core has the issue?

2023.9.3

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Smart Meter Texas

Link to integration documentation on our website

https://www.home-assistant.io/integrations/smart_meter_texas

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

@home-assistant
Copy link

Hey there @grahamwetzler, mind taking a look at this issue as it has been labeled with an integration (smart_meter_texas) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of smart_meter_texas can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign smart_meter_texas Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


smart_meter_texas documentation
smart_meter_texas source
(message by IssueLinks)

@excalbian
Copy link

I'm also seeing this issue since upgrading to HA 2023.9.3 (which might be a coincidence?). I haven't removed the integration, but I've tried disabling and re-enabling it a few times. In my logs I see:

ERROR (MainThread) [homeassistant.config_entries] Error setting up entry ****** for smart_meter_texas
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/smart_meter_texas/__init__.py", line 46, in async_setup_entry
    await smart_meter_texas_data.client.authenticate()
  File "/usr/local/lib/python3.11/site-packages/smart_meter_texas/__init__.py", line 218, in authenticate
    json_response = await resp.json()
                    ^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/aiohttp_client.py", line 72, in json
    return await super().json(*args, loads=loads, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1104, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://smartmetertexas.com/api/user/authenticate')

@Seamus-Mccartin
Copy link

The SMT website is showing a maintenance page for me.

Smart Meter Texas is currently undergoing maintenance. If further details are needed, please contact the help desk at 1-844-217-8595 .

@github-infohost-org
Copy link

Curious... i have been able to login and do on-demand reads from the website this entire time. My HA integration will not setup/initialize either. I hope SMT has not finally locked us out...

@Kage1
Copy link

Kage1 commented Oct 1, 2023

Looks like my last update was 9/29 which lines up when I upgraded to 2023.9.3.

Error log:

Logger: homeassistant.config_entries
Source: helpers/aiohttp_client.py:72
First occurred: September 30, 2023 at 10:30:50 AM (4 occurrences)
Last logged: 1:00:18 PM

Error setting up entry <name> for smart_meter_texas
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/smart_meter_texas/__init__.py", line 46, in async_setup_entry
    await smart_meter_texas_data.client.authenticate()
  File "/usr/local/lib/python3.11/site-packages/smart_meter_texas/__init__.py", line 218, in authenticate
    json_response = await resp.json()
                    ^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/aiohttp_client.py", line 72, in json
    return await super().json(*args, loads=loads, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1104, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://smartmetertexas.com/api/user/authenticate')

@logandirks
Copy link

I've come to report the same issue! Following post for ideas!

@github-infohost-org
Copy link

I had about 1.5 days worth of meter reads after updating to 2023.9.3 before it stopped working at 1am on 9/30, so i don't think it is related to HA version. I even downgraded to 2023.9.2 temporarily for a different problem and the integration would not initialize on that version either.

@randellhodges
Copy link
Contributor

Something wrong or changed with the smt website:

image

@pravardhanreddy
Copy link

Facing the same issue

Traceback (most recent call last):
  File "/lsiopy/lib/python3.11/site-packages/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.11/site-packages/homeassistant/components/smart_meter_texas/__init__.py", line 46, in async_setup_entry
    await smart_meter_texas_data.client.authenticate()
  File "/lsiopy/lib/python3.11/site-packages/smart_meter_texas/__init__.py", line 218, in authenticate
    json_response = await resp.json()
                    ^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.11/site-packages/homeassistant/helpers/aiohttp_client.py", line 72, in json
    return await super().json(*args, loads=loads, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/lsiopy/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1104, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://smartmetertexas.com/api/user/authenticate')

Following for updates

@lsellens
Copy link

lsellens commented Oct 2, 2023

Same issue here. Got my last read Sept 29th and nothing since.

Logger: homeassistant.config_entries
Source: helpers/aiohttp_client.py:72
First occurred: 9:21:54 PM (4 occurrences)
Last logged: 9:32:33 PM

Error setting up entry (redacted my username) for smart_meter_texas
Traceback (most recent call last):
  File "/home/homeassistant/.pyenv/versions/3.11.4/lib/python3.11/site-packages/homeassistant/config_entries.py", line 387, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.pyenv/versions/3.11.4/lib/python3.11/site-packages/homeassistant/components/smart_meter_texas/__init__.py", line 46, in async_setup_entry
    await smart_meter_texas_data.client.authenticate()
  File "/mnt/dietpi_userdata/homeassistant/deps/lib/python3.11/site-packages/smart_meter_texas/__init__.py", line 218, in authenticate
    json_response = await resp.json()
                    ^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.pyenv/versions/3.11.4/lib/python3.11/site-packages/homeassistant/helpers/aiohttp_client.py", line 72, in json
    return await super().json(*args, loads=loads, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/homeassistant/.pyenv/versions/3.11.4/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1104, in json
    raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://smartmetertexas.com/api/user/authenticate')

@Seamus-Mccartin
Copy link

Seamus-Mccartin commented Oct 2, 2023

I had about 1.5 days worth of meter reads after updating to 2023.9.3 before it stopped working at 1am on 9/30, so i don't think it is related to HA version. I even downgraded to 2023.9.2 temporarily for a different problem and the integration would not initialize on that version either.

I looked into it closer. https://smartmetertexas.com/ shows a maintenance page (and the API endpoint shows an nginx error), while https://www.smartmetertexas.com/ appears to work fine.

Can someone test whether the integration works with the new www domain?

A bit off-topic, but does anyone have a good workflow for importing SMT 15-minute interval reports over the 1 hour interval data that the integration provides?

@Mermarky
Copy link

Mermarky commented Oct 2, 2023

interesting.... I can confirm that "www.smartmetertexas.com" is working but "smartmetertexas.com" shows that the website is under maintenance. HOWEVER, the last read when you log in through "www" address shows Sept 30th only.
I'm running HA 2023.9.2 but I dont think the problem is HA related.

@Seamus-Mccartin
Copy link

the last read when you log in through "www" address shows Sept 30th only.

If that was under the "Latest End of Day Read" then that would be the correct information. It should tick over in ~1 hour. Using the "get current meter read" button I got a current reading that matches what I would expect if I went outside.

@chutson92
Copy link

Just confirming I have the same issue here. Deleted the integration and get the 'failed to connect' error for user authentication.

@Kage1
Copy link

Kage1 commented Oct 2, 2023

Looks like the owner of the add-on opened an issue for this. It appears the Smart Meter Texas site has updated their site causing SMT updates to fail. It is broken for everyone using SMT until the owner can get the connection to the API fixed.

grahamwetzler/smart-meter-texas#144

@Mermarky
Copy link

Mermarky commented Oct 2, 2023

If that was under the "Latest End of Day Read" then that would be the correct information. It should tick over in ~1 hour. Using the "get current meter read" button I got a current reading that matches what I would expect if I went outside.

Not for me. Tried to on SMT site "get current meter read" yesterday and today and there last reading is still Sept 30th

@jkubala4454
Copy link

I am having the same issue. Sept 30th was the last day of data.

@grahamwetzler
Copy link
Contributor

I'm afraid this might be the end of the road for this integration. The recent update to the SMT website protects the authentication endpoint with Akamai Bot Manager.

grahamwetzler/smart-meter-texas#144 (comment)

@Kage1
Copy link

Kage1 commented Oct 3, 2023

I'm afraid this might be the end of the road for this integration. The recent update to the SMT website protects the authentication endpoint with Akamai Bot Manager.

grahamwetzler/smart-meter-texas#144 (comment)

I was looking and there is a section in the user account for an API interface, but when I attempted to create a key I'm getting a message "API integration not done". Either I don't have rights to create one or they don't have it finished yet.

@grahamwetzler
Copy link
Contributor

The "API" is a joke. It seems to be targeted to energy companies.

In order to utilize the APIs you must provide 1) a Static IP Address and 2) a Signed Digital Certificate issued by a certificate authority and associated with the SMT registered company name

@Seamus-Mccartin
Copy link

The "API" is a joke. It seems to be targeted to energy companies.

In order to utilize the APIs you must provide 1) a Static IP Address and 2) a Signed Digital Certificate issued by a certificate authority and associated with the SMT registered company name

Just spitballing here but maybe nabu casa could help? If the home assistant cloud connection uses stable static IPs and they add a field to the letsencrypt certificates that might satisfy most of the requirements. I don't know if it's possible for integrations to proxy traffic like that through nabu casa though.

@mactizzle
Copy link

The "API" is a joke. It seems to be targeted to energy companies.

In order to utilize the APIs you must provide 1) a Static IP Address and 2) a Signed Digital Certificate issued by a certificate authority and associated with the SMT registered company name

@grahamwetzler I have the capability of providing a certificate and static IP to SMT long-term if that would be beneficial? I don't know how you are connecting with the API (I can't pretend to know python that well), but I'm thinking a sort of proxy that Texas folks could send requests through.

However, the "...associated with the SMT registered company name" part confuses me. Are they expecting a SAN on the certificate? IE, smt.domain.com ? If so, that's easy enough as I have a wildcard for my domain at work and the capability of spinning up a VM with a public IP that would never change.

Honestly doing it through nabu casa like @Seamus-Mccartin said is a good plan if they'll work with you on it, but I'm open to help if you hit a wall.

@Seamus-Mccartin
Copy link

Seamus-Mccartin commented Oct 6, 2023

If so, that's easy enough as I have a wildcard

apparently, they don't like wildcard certificates. But I think a SAN should be fine. The company name feels like it's more for organizations instead of individuals.

@ThermoMan
Copy link

I got an email from Smart Meter Texas recently (Thu, 21 Sept, 09:35) with a change to the TOS. New TOS here: https://www.smartmetertexas.com/security

Section E.2 may be the culprit here. Bold emphasis mine.

(2) monitor, gather, obtain, use, access or copy Content, Data, any information residing on any server or database connected to this Website, or any information of other users of this Website by using any robotic process automation, bot, spider, crawler, spyware, malware, beacon, engine, device, software, or extraction tool, other than as prescribed by identified functionality provided by this Website, or without legal authorization or permission; or

I don't know if that is new or not, but it seems aimed squarely at any automation type projects.

A possible work around might be to subscribe to a daily report using an email address and then giving HomeAssistant access to read that account and scrape the emails (it's easy for me to suggest how someone else can solve my problem since I don't know the details on any of this). I'd set up a new email account for this.

Also, they provide a way to contact them for help. Given that we are all paying customers of theirs, I've sent them an email asking for help...

Please Contact the SMT Helpdesk at 1-844-217-8595 or at support@smartmetertexas.com or Refer to Quick Reference Guide located at SMT Home Page

@grahamwetzler
Copy link
Contributor

Sorry everyone, even if the official API was a feasible option to replace the current integration, I'm not interested in working on a complete rewrite of this integration. Of course if someone else wants to tackle this, by all means, go for it. However, I will likely archive the grahamwetzler/smart-meter-texas repository shortly as I don't see this package being useful going forward.

@Kage1
Copy link

Kage1 commented Oct 7, 2023

Sorry everyone, even if the official API was a feasible option to replace the current integration, I'm not interested in working on a complete rewrite of this integration. Of course if someone else wants to tackle this, by all means, go for it. However, I will likely archive the grahamwetzler/smart-meter-texas repository shortly as I don't see this package being useful going forward.

@grahamwetzler It was good while it lasted. Thanks for the work on this integration.

@booyakafoo
Copy link

The NodeRed implementation I am using in my HA setup started working again today. I also noticed that my energy provider (Energy Texas) insights dashboard hasn't updated since Sept 23. It might be a coincidence but any chance that the changes SMT made also affected energy providers and they rolled back their changes?

@Kage1
Copy link

Kage1 commented Oct 7, 2023

Odd, I reloaded my integration and it didn't error out. Maybe?!

@ironjaw72
Copy link

Odd, I reloaded my integration and it didn't error out. Maybe?!

Same here...

@Kage1
Copy link

Kage1 commented Oct 7, 2023

@grahamwetzler It seems they made a change again, I'm now getting data from SMT again.

Edit: Well I got one update and so far no more besides that... :\

@lsellens
Copy link

lsellens commented Oct 8, 2023

I got a read too

@james-wiederkehr
Copy link
Author

I was also able to re-add the integration over the weekend.

@VandolinHimself
Copy link

Sorry everyone, even if the official API was a feasible option to replace the current integration, I'm not interested in working on a complete rewrite of this integration. Of course if someone else wants to tackle this, by all means, go for it. However, I will likely archive the grahamwetzler/smart-meter-texas repository shortly as I don't see this package being useful going forward.

@grahamwetzler It would seem the challenge manager at Akamai has changed. I've been able to scrape for about 24 hours now. Might not need archiving after all, but we'll see how the rest of the month goes.

@ironjaw72
Copy link

I experienced the original issue and then it started working as others reported back a couple of weeks ago.

However, this has resurfaced for me as of today. Watchman flagged the sensor and when I reloaded the integration I received the following which is the same as reported above.

Logger: homeassistant.config_entries
Source: config_entries.py:399
First occurred: 3:09:07 PM (1 occurrences)
Last logged: 3:09:07 PM

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 399, in async_setup
result = await component.async_setup_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/smart_meter_texas/init.py", line 46, in async_setup_entry
await smart_meter_texas_data.client.authenticate()
File "/usr/local/lib/python3.11/site-packages/smart_meter_texas/init.py", line 218, in authenticate
json_response = await resp.json()
^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/helpers/aiohttp_client.py", line 72, in json
return await super().json(*args, loads=loads, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client_reqrep.py", line 1105, in json
raise ContentTypeError(
aiohttp.client_exceptions.ContentTypeError: 0, message='Attempt to decode JSON with unexpected mimetype: text/html', url=URL('https://smartmetertexas.com/api/user/authenticate')

@lsellens
Copy link

Issue has come back for me too.

@NateEaton
Copy link

Same here... was fine then stopped Oct 20.

@christopher-keith240
Copy link

Unofficial SMT API is available again as of today.

@issue-triage-workflows
Copy link

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

@al-jimenez
Copy link

al-jimenez commented Feb 6, 2024

An update on my SMT integration: I had some response from this integration over the past month, but unfortunately it was very intermittent. I would reload the integration and sometimes it would restore the service for days, other times it would not. If I don't reload it then it stays OFF. Collected data is spotty because of the service outage.

I wanted to ask if it would be of benefit to do the following: create an script that would attempt 1 to 3 reloads 15 minutes apart. And set that up in a call from an automation at the beginning of the day. ¯_(ツ)_/¯ I do NOT want to create a denial of service issue with an automation and we lose this service permanently.

Any help would be appreciated. Thanks in advance.

THIS JUST CAME THROUGH ON SMT SITE: "SMT plans to deploy an upgrade that will require an Outage of all SMT services including the portal from Saturday February 10, 12:01 A.M. CST until Sunday February11, 10:30 P.M. CST. This should be a seamless upgrade to Smart Meter Texas users. If you have additional questions, please contact SMT website technical support at 1-844-217-8595" 🤞

@booyakafoo
Copy link

It's been working as well as it was prior to SMT making their maintenance changes for me. With that said it is spotty, but I believe it's an issue on SMTs side. When I get 0 kw/h results it's because the SMT current reading was the same as the previous reading an hour earlier.

@github-actions github-actions bot removed the stale label Feb 6, 2024
@al-jimenez
Copy link

It's been working as well as it was prior to SMT making their maintenance changes for me. With that said it is spotty, but I believe it's an issue on SMTs side. When I get 0 kw/h results it's because the SMT current reading was the same as the previous reading an hour earlier.

Yes, I agree its on SMT.

@jtsmagic
Copy link

Everything was working fine for me up until Jan 9. It hangs when I try to log in to SMT from the integration.

@misterfickey
Copy link

THIS JUST CAME THROUGH ON SMT SITE: "SMT plans to deploy an upgrade that will require an Outage of all SMT services including the portal from Saturday February 10, 12:01 A.M. CST until Sunday February11, 10:30 P.M. CST. This should be a seamless upgrade to Smart Meter Texas users. If you have additional questions, please contact SMT website technical support at 1-844-217-8595" 🤞

I haven't seen any energy data populate since February 9th, and I suspect this is the reason.

@wild-a
Copy link

wild-a commented Feb 21, 2024

There was an update to the Smart Meter Texas systems early January, after that finished, the integration stopped working for me. I wish home assistant could read the excel "subscriptions" that SMT generates and delivers via email.

@al-jimenez
Copy link

al-jimenez commented Mar 9, 2024

📌 As some status for everyone: In reviewing past posted issues I found this closed Issue: #88049; comment at the end says it all... bummer !:

This integration doesn't use the official API because it's incredibly onerous to get setup and use. This integration uses the same API that the website uses. It appears they implemented anti-bot protection which is something I don't expect we can work around.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.