Unable to send Push notification with content_available and mutable_content configurations in sygnal.yaml

[msl2000]

Hi all,

Whenever I add the fcm_options into my sygnal.yaml file,I am getting errors when a notification is being processed.

This is what my app config looks in the yaml file:

apps:
app.name:
type: gcm
api_version: v1
project_id:
service_account_file: /path/to/service_account.json
max_connections: 20
inflight_request_limit: 512
fcm_options:
content_available: true #These cause the issues
mutable_content: true #These cause the issues

These are the errors I am getting:

ERROR sygnal.gcmpushkin [1d40d8dd-a6d1-4656-a3db-706ac4606d97] 400 from server, we have sent something invalid! Error: '{\n "error": {\n "code": 400,\n "message": "Invalid JSON payload received. Unknown name \"content_available\" at 'message': Cannot find field.\nInvalid JSON payload received. Unknown name \"mutable_content\" at 'message': Cannot find field.",\n "status": "INVALID_ARGUMENT",\n "details": [\n {\n "@type": "type.googleapis.com/google.rpc.BadRequest",\n "fieldViolations": [\n {\n "field": "message",\n "description": "Invalid JSON payload received. Unknown name \"content_available\" at 'message': Cannot find field."\n },\n {\n "field": "message",\n "description": "Invalid JSON payload received. Unknown name \"mutable_content\" at 'message': Cannot find field."\n }\n ]\n }\n ]\n }\n}\n'

I can confirm that by removing those configurations, I am able to receive notifications - but I do need the mutable_content and content_available fields for iOS.

Any help would be much appreciated.

[MichalNemec]

Why was this closed? i just encountered the same issue, because i updated to v14.
can someone explain why that happens?

[devonh]

Would you be able to try changing the fcm_options config to use the following?

 fcm_options:
    apns:
      payload:
        aps:
          content-available: 1
          mutable-content: 1
          alert: ""

The new FCM api removed the top level APNS convenience fields and instead prefers them to be specifically declared inside of an apns block within the payload.
The alert field may or may not be necessary. There are various reports around the internet about it helping in some cases. I just don't have access to a convenient iOS setup to test it out right now.

If that works I can update the documentation to match.

[MichalNemec]

2024-04-08 21:50:56,705 [1] ERROR sygnal.http [8e1ff690-3ed5-4ac6-9b91-45643db56bea] Exception whilst dispatching notification.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sygnal/http.py", line 275, in _handle_dispatch
    result = await pushkin.dispatch_notification(notif, d, context)
  File "/usr/local/lib/python3.10/site-packages/sygnal/notifications.py", line 217, in dispatch_notification
    return await self._dispatch_notification_unlimited(n, device, context)
  File "/usr/local/lib/python3.10/site-packages/sygnal/gcmpushkin.py", line 520, in _dispatch_notification_unlimited
    data = GcmPushkin._build_data(n, device, self.api_version)
  File "/usr/local/lib/python3.10/site-packages/sygnal/gcmpushkin.py", line 673, in _build_data
    for attr, value in data["content"].items():
AttributeError: 'NoneType' object has no attribute 'items'
2024-04-08 21:50:57,669 [1] DEBUG sygnal.gcmpushkin [215c0bd6-5792-496c-99f8-2d98f3f0e3c1] GCM request took 0.107985 seconds
2024-04-08 21:50:57,669 [1] ERROR sygnal.gcmpushkin [215c0bd6-5792-496c-99f8-2d98f3f0e3c1] 400 from server, we have sent something invalid! Error: '{\n  "error": {\n    "code": 400,\n    "message": "Request contains an invalid argument.",\n    "status": "INVALID_ARGUMENT",\n    "details": [\n      {\n        "@type": "type.googleapis.com/google.firebase.fcm.v1.FcmError",\n        "errorCode": "INVALID_ARGUMENT"\n      },\n      {\n        "@type": "type.googleapis.com/google.rpc.BadRequest",\n        "fieldViolations": [\n          {\n            "field": "message.token",\n            "description": "Invalid registration token"\n          }\n        ]\n      }\n    ]\n  }\n}\n'
2024-04-08 21:50:57,670 [1] WARNING sygnal.http [215c0bd6-5792-496c-99f8-2d98f3f0e3c1] Failed to dispatch notification.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/sygnal/http.py", line 275, in _handle_dispatch
    result = await pushkin.dispatch_notification(notif, d, context)
  File "/usr/local/lib/python3.10/site-packages/sygnal/notifications.py", line 217, in dispatch_notification
    return await self._dispatch_notification_unlimited(n, device, context)
  File "/usr/local/lib/python3.10/site-packages/sygnal/gcmpushkin.py", line 574, in _dispatch_notification_unlimited
    new_failed, new_pushkeys = await self._request_dispatch(
  File "/usr/local/lib/python3.10/site-packages/sygnal/gcmpushkin.py", line 287, in _request_dispatch
    return self._handle_v1_response(
  File "/usr/local/lib/python3.10/site-packages/sygnal/gcmpushkin.py", line 431, in _handle_v1_response
    raise NotificationDispatchException("Invalid request")
sygnal.exceptions.NotificationDispatchException: Invalid request

did exactly this:

com.example.app:
    type: gcm
    api_version: v1
    project_id: projectID
    service_account_file: path/json
    fcm_options:
      apns:
        payload:
          aps:
            content_available: 1
            mutable_content: 1
            alert: ""

if i comment out entire fcm_options, its without any error.

[devonh]

Ah - that issue was fixed in #362. I'll make a new release with the fix, and then the new config changes I mentioned should work.

[devonh]

Would you be able to try with the 0.14.1 release?

[railsmechanic]

For me, it doesn't work either with version 0.14.1. Despite various attempts to configure content-available, in all the ways mentioned, the setup fails consistently, resulting in the following error message:

"Invalid JSON payload received. Unknown name \"content-available\" at 'message.data[0].value': Cannot find field."

Are there any other suggestions on how to configure it? We need the content-available setting to be True or 1 for our application to work correctly. We also need to use firebase for sending push notifications. Any ideas? 🤷‍♂️

[devonh]

For me, it doesn't work either with version 0.14.1. Despite various attempts to configure content-available, in all the ways mentioned, the setup fails consistently, resulting in the following error message:

"Invalid JSON payload received. Unknown name \"content-available\" at 'message.data[0].value': Cannot find field."

Are there any other suggestions on how to configure it? We need the content-available setting to be True or 1 for our application to work correctly. We also need to use firebase for sending push notifications. Any ideas? 🤷‍♂️

Would you be able to specify what your config looks like? I can't seem to reproduce that particular error you are seeing.

[railsmechanic]

Would you be able to specify what your config looks like? I can't seem to reproduce that particular error you are seeing.
This is the relevant part of the configuration, but I have already tested several variants:

apps:
  com.app.my:
    api_version: v1
    project_id: my-app
    service_account_file: /sygnal/my-app-firebase-adminsdk-sq3l9-deea15e111.json
    type: gcm
    fcm_options:
      apns:
        payload:
          aps:
            content-available: 1

[devonh]

hmm. Could you send the full log message from sygnal?

[railsmechanic]

This error is returned by sygnal:

2024-05-30 06:25:37,867 [1] INFO  sygnal.access Handled request: "10.42.14.209" - - [30/May/2024:06:25:37 +0000] "POST /_matrix/push/v1/notify HTTP/1.1" 502 - "-" "Synapse/1.105.1"
2024-05-30 06:25:54,162 [1] INFO  sygnal.gcmpushkin [6e5ec94f-acd7-4721-9e39-159404c20bcb] Sending (attempt 0) => ['<redacted>:<redacted>'] room:!ZygeAnECWdnSEuBfis:matrix.my-server.com, event:$veG0pHl06n55JIJwbhIpdlZXEiOd-gi8MjqPrunvSak
2024-05-30 06:25:54,272 [1] ERROR sygnal.gcmpushkin [6e5ec94f-acd7-4721-9e39-159404c20bcb] 400 from server, we have sent something invalid! Error: '{\n  "error": {\n    "code": 400,\n    "message": "Invalid JSON payload received. Unknown name \\"alert\\" at \'message.data[0].value\': Cannot find field.\\nInvalid JSON payload received. Unknown name \\"content-available\\" at \'message.data[0].value\': Cannot find field.\\nInvalid JSON payload received. Unknown name \\"mutable-content\\" at \'message.data[0].value\': Cannot find field.",\n    "status": "INVALID_ARGUMENT",\n    "details": [\n      {\n        "@type": "type.googleapis.com/google.rpc.BadRequest",\n        "fieldViolations": [\n          {\n            "field": "message.data[0].value",\n            "description": "Invalid JSON payload received. Unknown name \\"alert\\" at \'message.data[0].value\': Cannot find field."\n          },\n          {\n            "field": "message.data[0].value",\n            "description": "Invalid JSON payload received. Unknown name \\"content-available\\" at \'message.data[0].value\': Cannot find field."\n          },\n          {\n            "field": "message.data[0].value",\n            "description": "Invalid JSON payload received. Unknown name \\"mutable-content\\" at \'message.data[0].value\': Cannot find field."\n          }\n        ]\n      }\n    ]\n  }\n}\n'
2024-05-30 06:25:54,272 [1] WARNING sygnal.http [6e5ec94f-acd7-4721-9e39-159404c20bcb] Failed to dispatch notification.
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/sygnal/http.py", line 275, in _handle_dispatch
    result = await pushkin.dispatch_notification(notif, d, context)
  File "/usr/local/lib/python3.10/dist-packages/sygnal/notifications.py", line 217, in dispatch_notification
    return await self._dispatch_notification_unlimited(n, device, context)
  File "/usr/local/lib/python3.10/dist-packages/sygnal/gcmpushkin.py", line 574, in _dispatch_notification_unlimited
    new_failed, new_pushkeys = await self._request_dispatch(
  File "/usr/local/lib/python3.10/dist-packages/sygnal/gcmpushkin.py", line 287, in _request_dispatch
    return self._handle_v1_response(
  File "/usr/local/lib/python3.10/dist-packages/sygnal/gcmpushkin.py", line 431, in _handle_v1_response
    raise NotificationDispatchException("Invalid request")
sygnal.exceptions.NotificationDispatchException: Invalid request

[devonh]

This error is returned by sygnal:
...

Ok. So based off that error, I assume that you have specified something like this when calling POST /_matrix/client/r0/pushers/set:

"data": {
  "default_payload": {
    "aps": {
      "mutable-content": 1,
      "content-available": 1,
      "alert": {"loc-key": "SINGLE_UNREAD", "loc-args": []}
    }
  }
}

Sygnal takes that, and adds it into the message.data key of the FCM v1 notification.
This is what is causing the INVALID_ARGUMET error.

Removing the aps section from the default_payload should fix your issue.

[railsmechanic]

When removing the aps section it works with "silent notifications", only when the app is in foreground, but we need the "background update notification", which wakes up the app to process the notification.

Does the "content-available": 1 have to be set by calling POST - /_matrix/client/r0/pushers/set or in the configuration?

[devonh]

When removing the aps section it works with "silent notifications", only when the app is in foreground, but we need the "background update notification", which wakes up the app to process the notification.

Does the "content-available": 1 have to be set by calling POST - /_matrix/client/r0/pushers/set or in the configuration?

None of the aps config should be set using POST - /_matrix/client/r0/pushers/set when sending iOS notifications using the Firebase v1 API. Currently in sygnal, it should all be set using the config file. Using /pushers/set to configure aps will lead to INVALID_ARGUMENT errors from firebase in this scenario.

As for the issue with "silent notifications", I have found these two issues that contain potential workarounds.
https://stackoverflow.com/questions/37570200/firebase-silent-apns-notification/56165642#56165642
https://stackoverflow.com/questions/19239737/silent-push-notification-in-ios-7-does-not-work/

If you can try some of those suggestions out, I would very much appreciate an update on whether some combination of those configuration settings make things work.
Then I can update our documentation with suggested config setup.