Recent User-Agent Issues

[dereklucas]

Reading the recent pymyq fixes (#39 #40 #42) it seems clear that MyQ is blocklisting user agents. Initially, the user agent was a default from Python, then to something pulled from mobile Safari, then to a user agent pulled from the iOS app. Someone found that the Android app still uses the default string from the http library they use, but since they've shown a willingness to block one of their own user agents, I don't think we can simply continue pulling from new apps. Instead, we should either use a random string or myQ/[random number] CFNetwork/1107.1 Darwin/19.0.0 possibly randomized at a regular interval.

Unfortunately, if they view this as a problem eventually they will switch from a blocklist to a safelist and then we'll be sunk.

[dseven]

Hmmm. I had assumed that they were already whitelisting, but if appears that you are right - I tried a made up string, and it was allowed. I wonder if they're blacklisting as a means to force users to keep their apps updated, or as a way to keep people like us out......

I guess the risk is they can easily find this project and block whatever gets put in. I could understand if they still charged for premium integration subscriptions but it is a little weird they are doing it.....

[dereklucas]

I suspect that they either have already found this project, or they are looking at usage numbers and blocking anything that looks abnormally high. A randomly generated string that looks similar to the iOS app would keep us off of a high usage list and would probably require a safelist as a next step.

Using the Android user agent might also be a good option since in my experience it is more difficult to get all android users to update quickly. Also since they’re using a default string, it doesn’t look obvious that we’re using it.

It all depends on how likely it is that they’re intentionally targeting this project versus one person blocking things that look off.

[kyotodesertfox]

This product (MyQ) appears to be getting to a point now where this is no longer reliable to use anymore; I realize this is not a problem with HASS, etc, rather the dev's of MyQ.

Are there other garage door solutions that work well with Home Assistant, as I am on the verge of changing products. The amount of times this broke in the past month or so is just absurd, and honestly getting unacceptable. I know that we are doing as much as we can to keep up, but MyQ is just being d's now.

I cannot use the MyQ app either, as it constantly tells me the Sensor is broken; yet it has worked just fine using HASS, instead of the native app (which is amusing to me).

Sooo.....anyone know of any alternatives?

[rct]

1. I think making pymyq as indistinguishable from the Android app as possible is a good strategy as @dereklucas says, "it is more difficult to get all android users to update quickly". I guess at that point we will see if they are willing to force Android users to upgrade by cutting off access from the older apps.

2. How difficult would it be to allow the user agent to be overridden from Home Assistant config files instead of requiring a code change? (This might help until the "arms race" accelerates beyond user agent validation.

3. Anyway to judge how big the population of pymyq users is? Is it only used by Hass or do other HA platforms also use it? If I had known what the situation is, I never would have purchased anything from Chamberlain/Liftmaster.

[kyotodesertfox]

Anyway to judge how big the population of pymyq users is?

I wonder if HA Dev's can see these numbers on their end. Surely they monitor usage metrics like this?

If I had known what the situation is, I never would have purchased anything from Chamberlain/Liftmaster.

More true words were rarely spoken. I feel the same way.

[brbeaird]

Are you guys 100% sure user agent is required? I've been using my own integration for awhile now that does not send user agent at all.

[dereklucas]

@brbeaird It is not required. But, Python sets a default user agent. I’m not familiar enough with it to know how easily that can be disabled. But, if they’re willing to block specific user agents, it’s probably not significantly more work to block no user agent.

[brbeaird]

I do not think they're explicitly blocking certain agents. I think it is more that they use the agent string as a way to force clients to stay upgraded, and it seems if the agent isn't there, that logic is bypassed (for now at least).

[dseven]

Interesting. With no User-Agent specified in the headers, one of the form Python/3.7 aiohttp/3.6.2 gets added by default. This can be turned off by adding skip_auto_headers={'USER-AGENT'} to the request() call. With that in place, it does seem to be working (at least today).

[kyotodesertfox]

Wouldn't mind a PR ;-)

[dseven]

I can do a PR for the skip_auto_headers thing if there's a consensus that it's a good approach .. I guess I could put it up for review anyway....

[dseven]

master should be working currently