403 MismatchSenderId

[kayibal]

Any idea how to solve or debug this error? I'm using the latest version from master.

https://stackoverflow.com/questions/53875035/how-to-fix-403-mismatchsenderid-when-sending-web-push-notifications

I get the following traceback:

pytest test_push.py
====================================================== test session starts =======================================================
platform darwin -- Python 3.6.2, pytest-3.9.1, py-1.7.0, pluggy-0.8.0
rootdir: /Users/kayibal/Downloads, inifile:
plugins: django-3.4.3
collected 1 item

test_push.py F                                                                                                             [100%]

============================================================ FAILURES ============================================================
___________________________________________________________ test_push ____________________________________________________________

subs = {'endpoint': 'https://fcm.googleapis.com/fcm/send/cvNVGGLtZVo:APA91bFqfRXSHhqdzv6MXFKu7SFUvqyPSRlSNxER2B9cIj5OQZAC1THz...w1dNBzuNYFG7FMA', 'p256dh': 'BDz4O5Lb96W133iNj7uEmN0nnZuCDQKg8DTqa4P50stLUJ0vXBhwLker4EyMtf_U2Hr-UFf084QCxwZSR_3F70A'}}
vapid_data = {'privateKey': 'Ew2kli-56Ps6FEspgshs9MnFhhuX2mlMdXqhZqisN5w', 'publicKey': 'BEtyWjkXAXTOTN-5X018konhbR5KpAaQbM4jcWptLDzO2Ia-tm93NCY72TMh5kYAjYDThYY40FGh2BFHJeSX-04', 'subject': 'mailto:tech@rect.ag'}

    def test_push(subs, vapid_data):
        webpush(subs,
                'Your Push Payload Text',
                vapid_private_key=vapid_data['privateKey'],
                vapid_claims={"sub": "mailto:tech@rect.ag"},
>               ttl=2419200,
                )

test_push.py:27:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

subscription_info = {'endpoint': 'https://fcm.googleapis.com/fcm/send/cvNVGGLtZVo:APA91bFqfRXSHhqdzv6MXFKu7SFUvqyPSRlSNxER2B9cIj5OQZAC1THz...w1dNBzuNYFG7FMA', 'p256dh': 'BDz4O5Lb96W133iNj7uEmN0nnZuCDQKg8DTqa4P50stLUJ0vXBhwLker4EyMtf_U2Hr-UFf084QCxwZSR_3F70A'}}
data = 'Your Push Payload Text', vapid_private_key = 'Ew2kli-56Ps6FEspgshs9MnFhhuX2mlMdXqhZqisN5w'
vapid_claims = {'aud': 'https://fcm.googleapis.com', 'exp': 1548326836, 'sub': 'mailto:tech@rect.ag'}
content_encoding = 'aes128gcm', curl = False, timeout = None, ttl = 2419200

    def webpush(subscription_info,
                data=None,
                vapid_private_key=None,
                vapid_claims=None,
                content_encoding="aes128gcm",
                curl=False,
                timeout=None,
                ttl=0):
        """
            One call solution to endcode and send `data` to the endpoint
            contained in `subscription_info` using optional VAPID auth headers.

            in example:

            .. code-block:: python

            from pywebpush import python

            webpush(
                subscription_info={
                    "endpoint": "https://push.example.com/v1/abcd",
                    "keys": {"p256dh": "0123abcd...",
                             "auth": "001122..."}
                     },
                data="Mary had a little lamb, with a nice mint jelly",
                vapid_private_key="path/to/key.pem",
                vapid_claims={"sub": "YourNameHere@example.com"}
                )

            No additional method call is required. Any non-success will throw a
            `WebPushException`.

        :param subscription_info: Provided by the client call
        :type subscription_info: dict
        :param data: Serialized data to send
        :type data: str
        :param vapid_private_key: Vapid instance or path to vapid private key PEM \
                                  or encoded str
        :type vapid_private_key: Union[Vapid, str]
        :param vapid_claims: Dictionary of claims ('sub' required)
        :type vapid_claims: dict
        :param content_encoding: Optional content type string
        :type content_encoding: str
        :param curl: Return as "curl" string instead of sending
        :type curl: bool
        :param timeout: POST requests timeout
        :type timeout: float or tuple
        :param ttl: Time To Live
        :type ttl: int
        :return requests.Response or string

        """
        vapid_headers = None
        if vapid_claims:
            if not vapid_claims.get('aud'):
                url = urlparse(subscription_info.get('endpoint'))
                aud = "{}://{}".format(url.scheme, url.netloc)
                vapid_claims['aud'] = aud
            if not vapid_claims.get('exp'):
                # encryption lives for 12 hours
                vapid_claims['exp'] = int(time.time()) + (12 * 60 * 60)
            if not vapid_private_key:
                raise WebPushException("VAPID dict missing 'private_key'")
            if isinstance(vapid_private_key, Vapid):
                vv = vapid_private_key
            elif os.path.isfile(vapid_private_key):
                # Presume that key from file is handled correctly by
                # py_vapid.
                vv = Vapid.from_file(
                    private_key_file=vapid_private_key)  # pragma no cover
            else:
                vv = Vapid.from_string(private_key=vapid_private_key)
            vapid_headers = vv.sign(vapid_claims)
        response = WebPusher(subscription_info).send(
            data,
            vapid_headers,
            ttl=ttl,
            content_encoding=content_encoding,
            curl=curl,
            timeout=timeout,
        )
        if not curl and response.status_code > 202:
            raise WebPushException("Push failed: {} {}".format(
                response.status_code, response.reason),
>               response=response)
E           pywebpush.WebPushException: WebPushException: Push failed: 403 MismatchSenderId

[jrconlin]

Hmm. It's using Google's GCM version of WebPush so there are a few extra variables for the problem.

"SenderID" comes from GCM and usually is the registered ID for the Cloud application. IIRC, chrome provides a specific key based on the application that you use for the VAPID generation. I'm not 100% sure, but it may be that either the key you're using isn't the one that matches what Google thinks it should be, and it's telling you that the "SenderID" is failing.

I've not messed with Chrome's FCM system in a while so I don't know if things have changed internally.

[kayibal]

Thanks for the quick reply. So does pywebpush not support chrome browsers? Or am I just using it incorrectly? The interesting part is that it works in javascript with the same data...

[jrconlin]

It did the last time I checked, and I just re-ran a quick test script which accepted the chrome push message using my private key. I do note that there are some warnings from the EC library, but those didn't seem to impact the send. Guess I need to update that code. Looks like I was wrong about Google generating the VAPID private key, but there are a few things you should watch for. I did find https://developers.google.com/web/fundamentals/push-notifications/common-issues-and-reporting-bugs#authorization_issues which notes a few things that could cause the 401 message you're seeing, including mis-matched keys (which you said you already ruled out), an invalid expiration date for the JWT (I've seen this happen if you specify exactly 24 hours and your clock is off, try specifying an "exp" that is something more like `str(int(time.time()) + 43200)` That just means you'll have to use a new VAPID header every 12 hours instead of re-using the same one for an entire day. Or you could just generate a new VAPID header for each request and give it an EXP of time + 300 or something.) At least that's a few more items you can check to see if they fix your problem.

…

On Thu, Jan 24, 2019 at 3:47 AM Alan Höng ***@***.***> wrote: Thanks for the quick reply. So does pywebpush not support chrome browsers? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#100 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACLq8Ue92ExGFnAwF-P79v_csAAuVu9ks5vGZ1JgaJpZM4aPyX4> .

[jedi7]

Hi, I have the issue too (when used by Home Assistant)

Strange thing is it sometime works and I'm not still able to find out what is the change.
For example I unsubscribe, subscribe and then it works for some hours and after it, it starts sending "Error 403 (Forbidden)!!1"
But for example now, when I unsubscribe and subscribe, it does not work from start.

[jrconlin]

@jedi7 Are you getting the same 403 "MismatchSenderId" error? If so, that's really odd. Normally the SenderID shouldn't change for a given project, and is encoded in the endpoint URL that you've gotten.

It is possible for the WebPush system to tell your app that the subscription info has changed (see the pushsubscriptionchange event, but I would hope it wouldn't be every few hours, and would require your client app (like the Home Assistant UI or whatever you used to get the original subscription info) to get the update and re-subscribe.

[jedi7]

I'm getting just 403. (I'm using FCM vapid) See bellow:
I tried to keep the gcm sender id in Manifest, but there was no change.

About the event, thanks I will add it to my js script. But this will not be the issue. Because when I manually unsubscribe and subscribe, then it does not work. In other words it works only some time.

subscription:

{"endpoint":"https://fcm.googleapis.com/fcm/send/f0aVXEPkrkI:APA91bFCYMhBzkIByOE3OTohBts19N57Qq5lM075BdypX3X2Wf-OANuQcj8omVeIWDCsvtm1O5YE-D8cMxvPGfnIkVLw123E11ig9orDYrv6gWGN8GNt0bIFmESM16gW1FvvPZSs5n_c","expirationTime":null,"keys":{"p256dh":"BEknhUiwkmD9Zmc60bgjiqu3IhfbQPyL3EjsUFFj1iUj2ZrC43R3Q1cp2fY1SsRSF2CsjA7KBHYCn8YHpz3-k-c","auth":"XXXGn8fYgeePqb8e2SMnIA"}}

req headers: {'Authorization': 'WebPush eyJ0eXAiOi...', 'Crypto-Key': 'p256ecdsa=BN...'}_

response:

b'\n<!DOCTYPE html>\n<html lang=en>\n  <meta charset=utf-8>\n  <meta name=viewport content="initial-scale=1, minimum-scale=1, width=device-width">\n  <title>Error 403 (Forbidden)!!1</title>
\n <style>\n    *{margin:0;padding:0}html,code{font:15px/22px arial,sans-serif}html{background:#fff;color:#222;padding:15px}body{margin:7% auto 0;max-width:390px;min-height:180px;padding:30p
x 0 15px}* > body{background:url(//www.google.com/images/errors/robot.png) 100% 5px no-repeat;padding-right:205px}p{margin:11px 0 22px;overflow:hidden}ins{color:#777;text-decoration:none}a img{border:0}@media sc
reen and (max-width:772px){body{background:none;margin-top:0;max-width:none;padding-right:0}}#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54.png) no-repeat;margin-left:-5px}@media
only screen and (min-resolution:192dpi){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat 0% 0%/100% 100%;-moz-border-image:url(//www.google.com/images/logos/errorp
age/error_logo-150x54-2x.png) 0}}@media only screen and (-webkit-min-device-pixel-ratio:2){#logo{background:url(//www.google.com/images/logos/errorpage/error_logo-150x54-2x.png) no-repeat;-webkit-background-size
:100% 100%}}#logo{display:inline-block;height:54px;width:150px}\n  </style>\n  <a href=//www.google.com/><span id=logo aria-label=Google></span></a>\n  <p><b>403.</b> <ins>That\xe2\x80\x99s an error.</ins>\n  <p
>  <ins>That\xe2\x80\x99s all we know.</ins>\n'

[jedi7]

Interesting thing is. I just restarted the Home Assistant (which uses the pywebpush) and webpush started working.
Is possible the pywebpush is storing some persistent data which can make problems?

[jrconlin]

pywebpush shouldn't be doing any sort of storage or caching, so I'm a bit at a loss to know why it stopped working, and then started again. I know I'm probably grasping at straws, but I wonder if it might be a clock issue on the HomeAssistant box? If you're letting pywebpush generate the vapid header, if the clock is off that could cause the header to become invalid.

If you can get a failure, could you email me (jrconlin@gmail) the output of a --curl output so you've got a reproducable case?

[jedi7]

I just added some more debug logs and restarted HA. Now it is working, so I must wait some time to see the errors.
The clock on server is in sync. Maybe timezone is problem? Or the HA uses pywebpush wrongly. We will see.

[jedi7]

There was no change between these two push (only time)

==Working==

May  7 20:58:47 jwt token: claims: {'exp': 1557867527, 'nbf': 1557255527, 'iat': 1557255527, 'target': 'jare
k_pc', 'tag': 'b1a00025-d808-456e-b9bd-0545349fed68'}
payload={'badge': '/static/images/notification-badge.png', 'body': 'testovací zpráva', 'data': {'url': '/', 'jwt': '...'}, 'icon': '/static/icons/favicon-192x192.png', 'tag': 'b1a00025-d808-456e-b9bd-0545349fed68', 'title': 'myTitile', 'timestamp': 1557255527000}

==Nonworking (403)==

May  8 09:58:35 jwt token: claims: {'exp': 1557914315, 'nbf': 1557302315, 'iat': 1557302315, 'target': 'jare
k_pc', 'tag': 'd59073ed-c4bb-42fc-86b9-cbdc53d7f170'}
payload={'badge': '/static/images/notification-badge.png', 'body': 'testovací zpráva', 'data': {'url': '/', 'jwt': '....'}, 'icon': '/static/icons/favicon-192x192.png', 'tag': 'd59073ed-c4bb-42fc-86b9-cbdc53d7f170', 'title': 'myTitile', 'timestamp': 1557302315000}.

==creating jwt token===
jwt_token = jwt.encode(jwt_claims, jwt_secret).decode('utf-8')

After reboot of HomeAssistent then it started working again.
So, any ideas? Because I'm lost :(

[jedi7]

HA! I found it !
the problem is in the combination of calling webpush and HA.
Because webpush modifies his argument vapid_claims and add 'exp' if missing (it is set to +12h)
But when you call webpush(vapid_claims=self._vapid_claims) then the modification saves also into the class object. So next time of call webpush, the 'exp' is already in vapid_claims and is not updated.

I propose to make local copy of vapid_claims in webpush and then modify it.
What you think @jrconlin ?

[jurgenweber]

Looks like it is all related; home-assistant/core#23613

[jrconlin]

@jedi7 Ah, I think I see the bug. This is one of those python things that bites me every so often, isn't it? The one where passed structures are silently mutable.

I'll also add a patch to check and update the exp since I'm already goofing with it.

[Abrahram]

How to update the 'exp' ?

[jrconlin]

32e3fd9#diff-4896b0495b0608d9fbf022a4ed0acc7cR413

May want to check your local clock.