myQ: Unexpected error fetching myq devices data: Expecting ',' delimiter: line 4 column 66 (char 115)

[gmalbert]

The problem

MyQ is unresponsive within HA but the MyQ app is working, so I know that the service is up.

Environment

Home Assistant on Docker; Ubuntu 18.04; using HA version .111.4

• Home Assistant Core release with the issue:
• Last working Home Assistant Core release (if known):
• Operating environment (OS/Container/Supervised/Core):
• Integration causing this issue:
• Link to integration documentation on our website:

Problem-relevant configuration.yaml

None. Using integrations page to connect.

Traceback/Error logs

File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 136, in async_refresh
    self.data = await self._async_update_data()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 124, in _async_update_data
    return await self.update_method()
  File "/usr/local/lib/python3.7/site-packages/pymyq/api.py", line 149, in update_device_info
    "get", "Accounts/{0}/Devices".format(self.account_id), api_version=DEVICES_API_VERSION
  File "/usr/local/lib/python3.7/site-packages/pymyq/api.py", line 92, in request
    data = await resp.json(content_type=None)
  File "/usr/local/lib/python3.7/site-packages/aiohttp/client_reqrep.py", line 1040, in json
    return loads(stripped.decode(encoding))
  File "/usr/local/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.7/json/decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 4 column 66 (char 115)

Additional information

[probot-home-assistant]

myq documentation
myq source
_{^{(message by IssueLinks)}}

[broyuken]

Api changed yet again?

[brianjamescarter]

seems to me that Chamberlain is using this as a method to dissuade use of their API without a subscription. I am very close to replacing it a DIY solution that I know does not change :(
But hats off to the developers keeping this going! here and in pymyq!

[ChadGnad]

seems to me that Chamberlain is using this as a method to dissuade use of their API without a subscription. I am very close to replacing it a DIY solution that I know does not change :(
But hats off to the developers keeping this going! here and in pymyq!

I am with you on this, but I like what they are doing with the whole opening, open, closing, closed. The integration into HA is great with the icons too. I feel like a DIY would be easy with a wifi chip and a couple zigbee sensors. Does anyone know how the gateway works? Is it a bridge or a controller?

[bdraco]

If you have the myq 819LMB controller with homekit support, you can use homekit_controller instead https://www.home-assistant.io/integrations/homekit_controller/

No more cloud or api access needed

[ChadGnad]

so is this pretty much going to go on like this, or will they slow down....I guess my issue with it is that there should be an option to allow the local control. I would gladly give up MyQ for local integration into HA. I just don't want to add yet another hub. I am limiting myself to zigbee (thanks IKEA) and wifi. Not knowing how they are killing it, is this sustainable?

[bdraco]

so is this pretty much going to go on like this, or will they slow down...

You'll have to ask Chamberlain. I can only make guesses as to their intentions.

I guess my issue with it is that there should be an option to allow the local control.

The only local control option I know of that uses myq protocol is the 819LMB with homekit_controller

Home Assistant supports a lot of other Garage integrations that can be retrofitted onto an existing myq garage door opener https://www.home-assistant.io/integrations/#cover if the 819LMB isn't a fit.

[bowlingmh]

I got the same error, so I started just trying to access the MyQ api with curl. The problem is the User-Agent again. Just used the curl/7.68.0 User-Agent and all was well.

[broyuken]

If you have the myq 819LMB controller with homekit support, you can use homekit_controller instead https://www.home-assistant.io/integrations/homekit_controller/

No more cloud or api access needed

Do you need a HomeKit hub like an Apple TV or an iPad or does it connect directly to HA?

[galak]

Wonder about trying to have the user agent as config option so one could work around this locally until an update to the pymq and ha core are released.

[bowlingmh]

This seems like the best possible fix. Make all the default headers configurable so at least the breaking changes can be fixed fast.

…

On Fri, Jun 26, 2020 at 6:05 AM Kumar Gala ***@***.***> wrote: Wonder about trying to have the user agent as config option so one could work around this locally until an update to the pymq and ha core are released. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#37100 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABV6VPOUDYER5TIWP66JLFDRYSFHRANCNFSM4OIUBHSA> .

[GaryOkie]

Someone still has to determine what the new user agent header is after a change and post it. How is that done?

Would it be possible for pymyq to have a routine where it understands a connection failure occurs then tries to dynamically determine what the new user agent is and reconnects automatically?

That to me would be the best possible fix. Second best is a manually configurable header.

[mmotley999]

I looked at the pull request on pymyq. The one that seems to work is:

DEFAULT_USER_AGENT = "myQ/19569 CFNetwork/1107.1 Darwin/19.0.0"

I use a virtual environment '/srv/homeassistant'. In the '/srv/homeassistant/lib/python3.7/site-packages/pymyq' directory, edited 'api.py' and made the change. Save and restart HA.

It is now working.

[GaryOkie]

yep, that's what I did too. a little different technique with a standard Home Assistant install - but easily doable.

docker exec -t -i homeassistant /bin/bash
vi /usr/local/lib/python3.7/site-packages/pymyq/api.py

then change:
DEFAULT_USER_AGENT = "myQ/14041 CFNetwork/1107.1 Darwin/19.0.0"
to:
DEFAULT_USER_AGENT = "myQ/19569 CFNetwork/1107.1 Darwin/19.0.0"

save, and restart HA

[bowlingmh]

Where does this myQ "User-Agent" string come from? Why not just a generic curl version-number. Heck, it even works with a blank "User-Agent". Maybe that would be more robust to myQ backend changes?

[gavilaso]

Hi, i'm running HA OS on a raspberry pi.... i can't find that path at all.. i'm able to get to /lib/ but then i don't see anything inside that folder.. same think i try to go srv/ i don't see anything else.. i'm using putty to ssh.

any advise??

[galak]

Where does this myQ "User-Agent" string come from? Why not just a generic curl version-number. Heck, it even works with a blank "User-Agent". Maybe that would be more robust to myQ backend changes?

I was wondering something along these lines myself. I haven't looked into the details of how pymyq works.

[GaryOkie]

Hi, i'm running HA OS on a raspberry pi.... i can't find that path at all.. i'm able to get to /lib/ but then i don't see anything inside that folder.. same think i try to go srv/ i don't see anything else.. i'm using putty to ssh.

any advise??

You have to ssh to HA then run that docker command, otherwise you will not have access to the source files.

[bowlingmh]

If you want to play with authenticating on myQ, here's the curl command that will grab you an access token and the very command pymyq is failing with its current User-Agent. This command sends an empty User-Agent header and still works.

curl` -d '{"username": "YOURUSERNAME", "password": "YOURPASSWORD"}' -H "Content-Type: application/json" -H "MyQApplicationId: JVM/G9Nwih5BwKgNCjLxiFUQxQijAebyyg8QUHr7JOrP+tuPb8iHfRHKwTmDzHOu" -H "ApiVersion: 5.1" -H "BrandId: 2" -H "Culture: en" -H "User-Agent:" https://api.myqdevice.com/api/v5/Login  | python -m json.tool

Here's the broken call pymyq is doing for reference:

curl -d '{"username": "YOURUSERNAME", "password": "YOURPASSWORD"}' -H "Content-Type: application/json" -H "MyQApplicationId: JVM/G9Nwih5BwKgNCjLxiFUQxQijAebyyg8QUHr7JOrP+tuPb8iHfRHKwTmDzHOu" -H "ApiVersion: 5.1" -H "BrandId: 2" -H "Culture: en" -H "User-Agent: myQ/14041 CFNetwork/1107.1 Darwin/19.0.0" https://api.myqdevice.com/api/v5/Login  | python -m json.tool

You get the exact same error. If you want to see the malformed json response itself just drop the | python -m json.tool at the end of the command.

[gavilaso]

Hi, i'm running HA OS on a raspberry pi.... i can't find that path at all.. i'm able to get to /lib/ but then i don't see anything inside that folder.. same think i try to go srv/ i don't see anything else.. i'm using putty to ssh.
any advise??

You have to ssh to HA then run that docker command, otherwise you will not have access to the source files.

Perfect!! that did the Trick!!! Thanks for your assistant!

[octavioj]

Where is the directory for someone running on a virtual machine? I am running Home Assistant inside VirtualBox on a home server. I downloaded their packaged machine.

[dima-ser]

If you have the myq 819LMB controller with homekit support, you can use homekit_controller instead https://www.home-assistant.io/integrations/homekit_controller/

No more cloud or api access needed

Can you provide more details on this, how do I know if I have the 819LMB controller? How do I integrate it with HA?

[GaryOkie]

It's a separate MyQ Homebridge hub - and expensive at that.

They make good openers, but no way will I ever purchase anything from Chamberlain again.

[dwradcliffe]

You can buy it direct for (a little) cheaper: https://www.liftmaster.com/myq-home-bridge/p/G819LMB (or https://www.chamberlain.com/myq-home-bridge/p/MYQ-G0303-SP -- I believe they are both identical other than branding).
I'm considering this to remove this flaky api.

[dima-ser]

For $70 I'll just buy a local Z-wave opener, I'm done with MyQ.

…

On Fri, Jun 26, 2020, 13:42 David Radcliffe ***@***.***> wrote: You can buy it direct for (a little) cheaper: https://www.liftmaster.com/myq-home-bridge/p/G819LMB I'm considering this to remove this flaky api. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#37100 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABEFU2F3GYCZPSUMTRELRK3RYUB4FANCNFSM4OIUBHSA> .

[brandondougherty]

yep, that's what I did too. a little different technique with a standard Home Assistant install - but easily doable.

docker exec -t -i homeassistant /bin/bash
vi /usr/local/lib/python3.7/site-packages/pymyq/api.py

then change:
DEFAULT_USER_AGENT = "myQ/14041 CFNetwork/1107.1 Darwin/19.0.0"
to:
DEFAULT_USER_AGENT = "myQ/19569 CFNetwork/1107.1 Darwin/19.0.0"

save, and restart HA

This worked for me, i'm running docker on raspberry pi

[coreyPeterson]

Hi, I am new to HA. How do I make this change? I don't use docker or am not familar how to use it to adjust that file. I am running HA on a RPI 3B+ from the HA image.

[broyuken]

Wait for the next update and it will fix it for you.

[coreyPeterson]

thanks!

[dereklucas]

Reading the pymyq fixes (#39 #40 #42) it seems clear that MyQ is blocklisting user agents. Initially, the user agent was a default from Python, then to something pulled from mobile Safari, then to a user agent pulled from the iOS app. Someone found that the Android app still uses the default string from the http library they use, but since they've shown a willingness to block one of their own user agents, I don't think we can simply continue pulling from new apps. Instead, we should either use a random string or myQ/[random number] CFNetwork/1107.1 Darwin/19.0.0 possibly randomized at a regular interval.

Unfortunately, if they view this as a problem eventually they will switch from a blocklist to a safelist and then we'll be sunk. I'll crosspost this as a pymyq.

[broyuken]

Good call, if they do whitelist only then I will have to pick up that HomeKit bridge and be done with it once and for all.

[jer78]

@broyuken Don't give MyQ any more money with the "Homekit" bridge, it's a terrible piece of hardware. They had sent it to me for free since I had bought the MyQ bridge within a month or so of it coming out but it's complete garbage. It's a nightmare to pair and the support is terrible. In the end, I couldn't get it re-paired after something failed, spent hours on the phone with support, they had to escalate the issue to 2nd and 3rd level support but to no avail. They blamed it on the fact I was using an iPhone to pair. Umm...how can a HomeKit device not work with an iPhone??? Now I'm just relying on the home assistant integration. Once this is gone, I'm done with Chamberlain for good!