Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MyQ fails to setup with 401 or 429 error #101763

Closed
atruest opened this issue Oct 10, 2023 · 237 comments
Closed

MyQ fails to setup with 401 or 429 error #101763

atruest opened this issue Oct 10, 2023 · 237 comments

Comments

@atruest
Copy link

atruest commented Oct 10, 2023

The problem

HA 2023.10.1 worked ok for a few days but now similar failure when trying a reload. See error log below

What version of Home Assistant Core has the issue?

2023.10.1

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

MyQ

Link to integration documentation on our website

No response

Diagnostics information

Logger: homeassistant.components.myq
Source: helpers/update_coordinator.py:322
Integration: MyQ (documentation, issues)
First occurred: 10:19:24 AM (1 occurrences)
Last logged: 10:19:24 AM

Error fetching myq devices data: Error trying to re-authenticate to myQ service: Error requesting data from https://partner-identity.myq-cloud.com/connect/token: 401 - Unauthorized

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Same issue as before but now with 401 instead of 403

Additional information

No response

@sanctas
Copy link

sanctas commented Oct 10, 2023

I am having the same issue as well. I recently hooked an ESP32 up to my garage door opener and added 2 reed switches to the garage door so I can control it locally. Very glad I did it.

@brad07x
Copy link

brad07x commented Oct 10, 2023

Confirming similar behavior and the same error returned from MyQ after reloading HA when the integration is starting. Was working OK since the previous fix was merged until breaking again this morning.

Logger: pymyq.api
Source: components/myq/__init__.py:42
First occurred: 11:30:30 AM (5 occurrences)
Last logged: 11:33:24 AM
Authentication failed: Error requesting data from https://partner-identity.myq-cloud.com/connect/token: 401 - Unauthorized

Similar to @sanctas, I have a ratgdo (https://paulwieland.github.io/ratgdo/) kit on order for local control of my Security+ 2.0 opener, but it would be helpful to have this integration working again until my backordered unit ships out. Thanks all!

EDIT: Wanted to add that the MyQ app on android is working without issues from the same network/internet connection used by my HA instance.

@sanctas
Copy link

sanctas commented Oct 10, 2023

Similar to @sanctas, I have a ratgdo (https://paulwieland.github.io/ratgdo/) kit on order for local control of my Security+ 2.0 opener, but it would be helpful to have this integration working again until my backordered unit ships out. Thanks all!

I actually used an ESP32 soldered to my wireless garage door opener. I'd use the Ratgdo but it's backordered so I went this route instead.

@traveler2100
Copy link

I lost connect in the last 30 minutes due to 429 Too Many Request but phone app still works:

Config entry 'xxx@xxx.com' for myq integration not ready yet: Got 429 error - stopping request until 2023-10-10 16:30:38.802025. there were 1 request; Retrying in background

@luisiam
Copy link

luisiam commented Oct 10, 2023

Getting 401 starting from 8:14AM (PDT). Ordered ratgdo and waiting for it to be arrived.

@home-assistant
Copy link

Hey there @ehendrix23, @Lash-L, mind taking a look at this issue as it has been labeled with an integration (myq) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of myq can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign myq Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)


myq documentation
myq source
(message by IssueLinks)

@justinlindh
Copy link
Contributor

justinlindh commented Oct 10, 2023

I actually used an ESP32 soldered to my wireless garage door opener.

Do you have a link handy explaining how this gets wired up/flashed? I'd like to do the same.

Edit: Found this. It should work for me.

@barqers
Copy link

barqers commented Oct 10, 2023

Just started geting this error - first it was the 429 error, deleted the integration and waited til the specified time, then tried re-adding it again and now get 401 unauthorized.

@Lash-L
Copy link
Contributor

Lash-L commented Oct 10, 2023

@home-assistant rename MyQ fails to setup with 401 or 429 error

@home-assistant home-assistant bot changed the title MyQ failed again. Now 401 MyQ fails to setup with 401 or 429 error Oct 10, 2023
@Lash-L
Copy link
Contributor

Lash-L commented Oct 10, 2023

As a reminder - please upvote the original post if you are having the same problem, do not add extra comments stating that you have the same problem - instead upvote the original post. I get buried in notifications and it is extremely difficult to separate helpful information and just chatter.

I don’t have a ton of time this week to try to look at it, but I will try my best. As a reminder to everyone - I don’t own one of these devices, and I am always looking for a new codeowner, I have had one user step forward and attempt to diagnose some problems, I will give them as much support as I can with my limited time this week and if they aren’t able to figure it out, I will try to solve what I can with my limited time

@pawelrozek
Copy link

It doesn't seem to be specific to HomeAssistant version. I rolled back to 2023.9.3 and the issue persists.

@Smkyb77
Copy link

Smkyb77 commented Oct 10, 2023

I am on 2023.9.2 but the latest MyQ code and it is failing

@drysart
Copy link

drysart commented Oct 10, 2023

Fixed by updating api.py so that the call to get the OAuth token more closely represents a valid OAuth token call. Changes highlighted in yellow:

image

  • Add an authorization header to specify the client ID (base64 encoded value for IOS_CGI_MYQ:)
  • Add an Accept header with */*
  • Remove the unnecessary client_id, client_secret, and scope form fields since they're not supposed to be sent on the get token call in the first place
  • Added [0] to the line extracting the code form field to get the code as a string rather than as an array

As text:

            resp, data = await self.request(
                returns="json",
                method="post",
                url=OAUTH_TOKEN_URI,
                websession=session,
                headers={
                    "Authorization": "Basic SU9TX0NHSV9NWVE6",
                    "Content-Type": "application/x-www-form-urlencoded",
                    "Accept": "*/*",
                },
                data={
                    # "client_id": OAUTH_CLIENT_ID,
                    # "client_secret": OAUTH_CLIENT_SECRET,
                    "code": parse_qs(urlsplit(redirect_url).query).get("code", "")[0],
                    "code_verifier": self._code_verifier,
                    "grant_type": "authorization_code",
                    "redirect_uri": OAUTH_REDIRECT_URI
                    # "scope": parse_qs(urlsplit(redirect_url).query).get(
                    #     "code", "MyQ_Residential offline_access"
                    # ),
                },
                login_request=True,
            )

@drysart
Copy link

drysart commented Oct 10, 2023

And as an aside, it seems that MyQ is slowly tightening their API validations; the code needs a thorough once-over to make sure it's actually compliant with OAuth properly and it isn't doing any other extraneous/incorrect stuff; because given the history of this integration over the past month or so, any further deviations from the spec lurking in the code are likely to result in additional breakage in the future.

And also to add, if you apply this fix you may continue to receive 429 errors until your rate limiting on the MyQ API (caused by the broken code previously running and having sent too many requests to the API and getting your IP temporarily blocked for a while). Those issues should clear up within an hour or two.

@Lash-L
Copy link
Contributor

Lash-L commented Oct 10, 2023

Hey @drysart Thanks for your insight. That's rather interesting - it is my understanding that basically everything the maintainers before me have in this library is just a port from the homebridge myq api. They use scope, client_id, secret, etc.

I have a dummy account with no devices that I have used for testing for logging in before - I can give this a try in a bit and see if I get similar results. Where do you get your "Basic SU9TX0NHSV9NWVE6" from?

Feel free to make a PR on the python-myq repo I'd be more than happy to directly merge in your changes if it works for me as well (If you don't want to for any reason - I can make the changes later)

@justinlindh
Copy link
Contributor

Where do you get your "Basic SU9TX0NHSV9NWVE6" from?

It's just base64 encoded IOS_CGI_MYQ: so I assume it's what the IOS app uses.

I applied drysart's change and am still throttled, but maybe it just needs to clear.

@Smkyb77
Copy link

Smkyb77 commented Oct 10, 2023

Can confirm after changes and after the timeout expired I was able to authenticate correctly. The interesting thing now is to see if we get 429's since it is calling more frequently than once per minute.

@BeerDiet
Copy link

BeerDiet commented Oct 10, 2023

Trying this now... will report back.

Edit: Still getting 429s, will wait another hour or 2.

Edit edit: Confirmed. Nice work, @drysart!

@drysart
Copy link

drysart commented Oct 10, 2023

It's just base64 encoded IOS_CGI_MYQ: so I assume it's what the IOS app uses.

Most likely. I grabbed it from a Postman login request to MyQ.

Can confirm after changes and after the timeout expired I was able to authenticate correctly. The interesting thing now is to see if we get 429's since it is calling more frequently than once per minute.

I don't think it will. The 429s were arising in the first place because the login logic basically spams the service with retries when the login fails (it retries 5 times, then homeassistant restarts the integration which causes it to do it again right away). Without that issue, the API usage should be well below the rate limits.

@justinlindh
Copy link
Contributor

Can confirm that @drysart 's fix did resolve this, once my timeout expired. Has been solid for 10 minutes. Will report back if it loses availability. Thanks!

@zander96
Copy link

I have tested @drysart ‘s changes and can confirm they are working for me and my HA instance is authenticating correctly. Thank you!!

I had disabled the (broken) integration probably 4-5 hours ago when I noticed it wasn’t working, and I didn’t get a 429 error when I tried the changes just now, so if you try and it doesn’t work, try again in a couple hours.

@kmurrietta
Copy link

kmurrietta commented Oct 10, 2023

Just wanted to confirm that @drysart 's changes are working for me as well. I would suggest anyone having issues with this to disable able the integration until you get an update to avoid any 429 Rate limit errors

Also, Thank you!!

@atruest
Copy link
Author

atruest commented Oct 10, 2023 via email

@drysart
Copy link

drysart commented Oct 10, 2023

@Lash-L See Python-MyQ/Python-MyQ#18

@neydah700
Copy link

@drysart updated api.py working for me! Thanks!

@kevinkorte
Copy link

Also got hit with the 429 error about 3 hours ago.

Logger: homeassistant.components.myq
Source: helpers/update_coordinator.py:322
Integration: MyQ (documentation, issues)
First occurred: 9:58:49 AM (1 occurrences)
Last logged: 9:58:49 AM
Error fetching myq devices data: Got 429 error - stopping request until 2023-10-18 17:43:49.017082. there were 30367 request

@joeshaw
Copy link

joeshaw commented Oct 18, 2023

The latest batch of 429 errors are from the https://partner-identity.myq-cloud.com/connect/authorize endpoint and appear to be Cloudflare anti-bot challenges, which require cookies and javascript to get past. It might be necessary to kick out to a browser to get past these.

@x046866x
Copy link

x046866x commented Oct 18, 2023 via email

@x046866x
Copy link

x046866x commented Oct 18, 2023 via email

@jermanoid
Copy link

jermanoid commented Oct 19, 2023

FYI go ahead and disable your MyQ addons until this is resolved.

@jeeftor
Copy link
Contributor

jeeftor commented Oct 19, 2023

How come this doesn't trigger on the phone app? Is there some cert or something embedded in the app?

@joeshaw
Copy link

joeshaw commented Oct 19, 2023

@jeeftor The login process runs a browser which gets through the Cloudflare bot check.

@RyTheITGuy
Copy link

RyTheITGuy commented Oct 19, 2023

I'm seeing the 429 errors as well in the last day or two. Not sure if this is helpful:

10/19/2023, 8:38:13 AM Bridge myQ Configuration change
10/19/2023, 8:38:13 AM Bridge myQ bridgemyq stopping
10/19/2023, 8:38:13 AM Bridge myQ myQ Garage [ ERROR ] myQ API error: Temporary myQ API server-side issues encountered: 429 - Too Many Requests. This typically indicates a myQ API lockout for a 60-90 minute period before resuming API connectivity.
10/19/2023, 8:38:18 AM Bridge myQ Bridge myQ starting
10/19/2023, 8:38:18 AM Bridge myQ Loaded plugin 'homebridge-myq'
10/19/2023, 8:38:18 AM Bridge myQ [ DEBUG ] Registering platform 'homebridge-myq.myQ'
10/19/2023, 8:38:18 AM Bridge myQ Loading 1 platforms...
10/19/2023, 8:38:18 AM Bridge myQ myQ Garage Debug logging on. Expect a lot of data.
10/19/2023, 8:38:18 AM Bridge myQ Bridge is running on port 51856.
10/19/2023, 8:38:19 AM Bridge myQ myQ Garage myQ API: Temporary myQ API server-side issues encountered: 429 - Too Many Requests. This typically indicates a myQ API lockout for a 60-90 minute period before resuming API connectivity. Retrying the API call.
10/19/2023, 8:38:19 AM Bridge myQ myQ Garage myQ API: Switching to myQ cloud region: east.
10/19/2023, 8:38:19 AM Bridge myQ myQ Garage myQ API: Temporary myQ API server-side issues encountered: 429 - Too Many Requests. This typically indicates a myQ API lockout for a 60-90 minute period before resuming API connectivity. Retrying the API call.
10/19/2023, 8:38:19 AM Bridge myQ myQ Garage myQ API: Switching to myQ cloud region: west.
10/19/2023, 8:38:19 AM Bridge myQ myQ Garage myQ API: Temporary myQ API server-side issues encountered: 429 - Too Many Requests. This typically indicates a myQ API lockout for a 60-90 minute period before resuming API connectivity. Retrying the API call.
10/19/2023, 8:38:19 AM Bridge myQ myQ Garage myQ API: Switching to myQ cloud region: auto.

I've removed and reinstalled the bridge without success. I've stopped it for now.

@crizz6396
Copy link

Is it possible to connect to a different service such as total connect and control.from there. I have TC already connected. Why can't we pull in the the other devices such as myq

@mstberto
Copy link

Mine was not working yesterday but appears to be working again today.

@dmperl
Copy link

dmperl commented Oct 20, 2023

@mstberto same for me.. I disabled the integration when the "problem" reappeared on October 17. I just saw your post, re-enabled the MyQ integration. I'm back in business. until the next time ;-)

edit 1: (one day later) it's back!!! 429 errors. I've disable the MyQ integration and ordered 2 of these: https://www.athom.tech/blank-1/garage-door-opener-for-esphome ... nice packaging, pre-flashed, No blinking warning light when closing.

I'll follow along to see how this saga turns out, but my garage doors will work reliably from HA in the meantime.

@ThatTallGuy21
Copy link

On 10.3.. I never disabled mine to begin with and it was failing. Still failed when I tried restarting the integration. Disabled, restarted HA, and re-enabled and it's working now.

@devlinford
Copy link

Hi all!

It sounds like some of you have working systems as of today? I just noticed this issue today and I am still experiencing it

Running Homebridge 1.6.1
Homebridge myQ v3.4.3

Whenever I turn on myQ I get this every two mins:

[10/20/2023, 7:01:05 PM] [Garage Doors] myQ API error: Temporary myQ API server-side issues encountered: 429 - Too Many Requests. This typically indicates a myQ API lockout for a 60-90 minute period before resuming API connectivity.

Notably, if I rollback to Homebridge myQ v3.4.2, I get this every two mins:

[10/20/2023, 7:05:22 PM] [Garage Doors] myQ API error: Temporary myQ API server-side issues encountered: 429 - Too Many Requests.

Does anyone know what exactly I need to do to get the latest version working again?

Thanks!

@RyTheITGuy
Copy link

I removed, reinstalled, and changed my myQ password to no avail. Still getting same 429 errors.

Bridge myQ v3.4.3
Homebridge 1.6.0

@nytron88
Copy link

I performed the same steps as @RyTheITGuy and have the same issues

@AlekEagle
Copy link

H.A. 2023.10.3
H.A. Supervisor 2023.10.0
O.S. 11.0

Error 429s here too, I guess they're starting to get really picky with who does and doesn't have access to the API?

If sniffing the mobile app's network traffic would be helpful I can try and investigate that.

@hitnrun30
Copy link

H.A. 2023.10.3 H.A. Supervisor 2023.10.0 O.S. 11.0

Error 429s here too, I guess they're starting to get really picky with who does and doesn't have access to the API?

If sniffing the mobile app's network traffic would be helpful I can try and investigate that.

They started throttling calls to get status unless the originator comes from an affiliate

@crizz6396
Copy link

This makes no sense. When I install the pymyq folder in my custom components I can sign in and get the 429 error. When I run the installation from the internal folder I get failed to connect. Is there a way to authenticate externally through a web browser?

@drysart
Copy link

drysart commented Oct 22, 2023

I wouldn't draw any conclusions from testing like that. They have Cloudflare's bot protection on pretty sharply, so as soon as you start getting 429 errors, all bets are off until the heat on you cools down. The 429s aren't really traditional rate limiting; it's Cloudflare detecting a potentially automated access and throwing up a challenge page with a 429 status code.

There isn't really a good workaround for that for Home Assistant, because even if you do an initial authentication request from a browser and grab some cookies to plug into HA (like some other integrations do), those cookies are only good for a relatively short period of time before Cloudflare will start challenging again; so unless you want to be authenticating manually every few hours it's not worth doing.

Honestly, at this point, just tell everyone you know not to buy any MyQ devices, give them one star reviews on every online storefront you can find; and then wire up a Shelly or a ratdgo or some other roll-your-own solution and give up on using MyQ automation entirely. Short of a significant policy change with MyQ management, that's going to be the only reliable solution.

@crizz6396
Copy link

I wouldn't draw any conclusions from testing like that. They have Cloudflare's bot protection on pretty sharply, so as soon as you start getting 429 errors, all bets are off until the heat on you cools down. The 429s aren't really traditional rate limiting; it's Cloudflare detecting a potentially automated access and throwing up a challenge page with a 429 status code.

There isn't really a good workaround for that for Home Assistant, because even if you do an initial authentication request from a browser and grab some cookies to plug into HA (like some other integrations do), those cookies are only good for a relatively short period of time before Cloudflare will start challenging again; so unless you want to be authenticating manually every few hours it's not worth doing.

Honestly, at this point, just tell everyone you know not to buy any MyQ devices, give them one star reviews on every online storefront you can find; and then wire up a Shelly or a ratdgo or some other roll-your-own solution and give up on using MyQ automation entirely. Short of a significant policy change with MyQ management, that's going to be the only reliable solution.

Is there a way to piggy back off an authenticated system. For example I have totally connect. They are connected through that.

@x046866x
Copy link

x046866x commented Oct 22, 2023 via email

@traveler2100
Copy link

When I upgraded my doors I did not realize the ISY994 sensor I used for years would not work any longer because the contacts are not available on the opener. I can implement some other local solution as above but one of my main uses for the MyQ integration is detecting the two garage doors on a second house not on the local network. Something I cannot do with a local solution. Its hard to understand why MyQ engineers and managers do not see the importance of providing a solution for the growing open source smart home market. I guess they want control.

@MikeMcGlinch
Copy link

> When I upgraded my doors I did not realize the ISY994 sensor I used for years would not work any longer because the contacts are not available on the opener. I can implement some other local solution as above but one of my main uses for the MyQ integration is detecting the two garage doors on a second house not on the local network. Something I cannot do with a local solution. Its hard to understand why MyQ engineers and managers do not see the importance of providing a solution for the growing open source smart home market. I guess they want control.

And $$$$$$!~ (Cha Ching!!) But to be so shortsighted.....? I guess taking advantage of an already entrenched (loyal?) customer base for ongoing revenue streams trumps (and I mean that word with all the irony it carries) any beneficial goodwill to this community. They see the $$ opportunity and grab, as more of this market is tending to do. I myself don't hold out hope for Matter to make things any better.

@JanVanHyfte
Copy link

In fact, I am using another integration (https://github.com/brbeaird/SmartThings-MyQ-Edge) in a docker container that "was" also suffering 429's. But some hours ago, the thing came alive ...

@jermanoid
Copy link

> When I upgraded my doors I did not realize the ISY994 sensor I used for years would not work any longer because the contacts are not available on the opener. I can implement some other local solution as above but one of my main uses for the MyQ integration is detecting the two garage doors on a second house not on the local network. Something I cannot do with a local solution. Its hard to understand why MyQ engineers and managers do not see the importance of providing a solution for the growing open source smart home market. I guess they want control.

And $$$$$$!~ (Cha Ching!!) But to be so shortsighted.....? I guess taking advantage of an already entrenched (loyal?) customer base for ongoing revenue streams trumps (and I mean that word with all the irony it carries) any beneficial goodwill to this community. They see the $$ opportunity and grab, as more of this market is tending to do. I myself don't hold out hope for Matter to make things any better.

Blackrock bought them a couple years ago, which explains most of the behavior.

@dmperl
Copy link

dmperl commented Oct 23, 2023

@JanVanHyfte seeing your post I re-enabled the MyQ integration. Surprisingly it works again. I'll leave it enabled until it goes bad again. Since I have a 10 year old door opener I am going to try this: https://www.reddit.com/r/homeassistant/comments/17d2sq6/comment/k5u3zt7/?utm_source=share&utm_medium=web2x&context=3

Edit 1: It was good for about 15 minutes... back to 429 errors. Disable the integration again. End of experiment.

@Lash-L
Copy link
Contributor

Lash-L commented Oct 23, 2023

I have an update here: https://community.home-assistant.io/t/the-current-state-of-myq-from-the-codeowner/630623
I am closing and locking this issue - to only have one github issue open for this problem - please see #101912

@home-assistant home-assistant locked and limited conversation to collaborators Oct 23, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests