Add messaging send_all and send_multicast functions

[ZachOrr]

Hey y'all - I'm working on adding the sendAll and sendMulticast features for #196 based on firebase/firebase-admin-node#453 - I was hoping I could get some assistance with how to represent returns from these functions, since the current send endpoint will raise if the response from FCM contains an error, and this doesn't seem like a great solution when sending several requests.

Also, if I could get suggestions on a non-regex way to wrangle batch response content, that would be appreciated. My first thought was seeing if we could dump each individual response content in to a requests Response object, but that doesn't appear to be a thing with Response objects.

[hiranya911]

Thanks @ZachOrr for working on this. We actually have an approved API design for this. It goes something like this:

firebase_admin.messaging
  send_all(messages, dry_run=False, app=None): BatchResponse
  send_multicast(multicast_message, dry_run=False, app=None): BatchResponse

  class MulticastMessage
    tokens: list<str>
    Everything else on Message class except token, topic and condition

  class BatchResponse
    responses: list<SendResponse>
    success_count: int
    failure_count: int

  class SendResponse
    success: boolean
    message_id: str
    exception: ApiCallError

You might want to look into using the Google API client for making the batch requests: https://developers.google.com/api-client-library/python/guide/batch

[ZachOrr]

@hiranya911 Thanks for the guidance! I think I'm getting closer on this one. I left some comments with specific questions. Could you take a look when you get a chance?

[hiranya911]

Hi @ZachOrr. From what I can tell googleapiclient and googe.auth do not play well with each other in Python. This is unfortunate, as they really work well in other languages like Java and C#. In any case, I managed to send a batch request using the following code:

import json

import googleapiclient
from googleapiclient import http
from googleapiclient import _auth
import google.auth

# Init Google Credentials (this part is currently handled by firebase_admin.credentials)
_scopes = [
    'https://www.googleapis.com/auth/cloud-platform',
    'https://www.googleapis.com/auth/datastore',
    'https://www.googleapis.com/auth/devstorage.read_write',
    'https://www.googleapis.com/auth/firebase',
    'https://www.googleapis.com/auth/identitytoolkit',
    'https://www.googleapis.com/auth/userinfo.email'
]
creds, _ = google.auth.default(scopes=_scopes)

# It sucks that we have to use this internal _auth module. Wish there was a better way
transport = _auth.authorized_http(creds)

responses = []

def callback(req_id, resp, exception):
  print(req_id, resp, exception)
  responses.append(resp['name'])

def postproc(resp, body):
  """This gets called for each part. It's a low-level function where we can parse the HTTP response."""
  if resp.status == 200:
    return json.loads(body)
  else:
    raise Exception('unexpected response')

batch = http.BatchHttpRequest(callback, 'https://fcm.googleapis.com/batch')

body = json.dumps({'message': {'topic': 'foo'}})
req = http.HttpRequest(
  http=transport, postproc=postproc,
  uri='https://fcm.googleapis.com/v1/projects/my-project-id/messages:send',
  method='POST',
  body=body)

# Queue 3 requests in the batch
batch.add(req)
batch.add(req)
batch.add(req)

batch.execute()
print(responses) # Prints 3 message IDs

[ZachOrr]

Awesome! Thanks for the help. It seems like using app.credential.get_credential() in _auth.authorized_http works fine - can we do that?

[hiranya911]

@ZachOrr yes, lets do that for now. I'll see if we can find a better alternative for that part.

[ZachOrr]

Added tests and added errors raises from batch request http errors.

[ZachOrr]

Not entirely sure what the fix is to get that json.loads working on some versions of Python. Any suggestions?

[hiranya911]

I think this is shaping up pretty nicely. As for the test failure, it seems the response payload is coming through as a bytes object. So we'll need to decode it first.

json.loads(response.decode())

[hiranya911]

I used this in my hack. But we need to raise something more detailed here. I think we need to parse the error response body, and construct a messaging.ApiCallError.

[ZachOrr]

Is there a situation where FCM will return a non-200 or non-error response code? I'm trying to figure out what to do here. A messaging.ApiCallError takes an error, which we really don't have in this case.

[hiranya911]

Technically it can be any HTTP response. But in practice FCM mostly returns 200 or 4xx/5xx error codes. We can treat anything that is not 200 as an error.

We need to somehow call the error parsing logic in _handle_fcm_error() on this, and get an ApiCallError out of it. May be refactor the parsing logic into a new helper function:

if resp.status != 200:
  code, message = _parse_error_response(response)
  raise ApiCallError(code, message) # last argument is optional  

[hiranya911]

Resolves #277

[ZachOrr]

Alright - I think this is ready for a re-review.

[hiranya911]

Thanks @ZachOrr. I think something is broken somewhere. There are a bunch of tests where the API raises exceptions, where it should not. Looks like the code is not handling some case correctly.

[hiranya911]

Now I'm confused. Why wouldn't this throw? It is the right behavior, but how is this any different from the test cases above?

[ZachOrr]

I think I have everything addressed so far.

[hiranya911]

Thanks @ZachOrr. Looks good. Please fix the lint error detected by CI, and then I can approve/merge this.

You can add delete resp to the method to deal with unused args.

[ZachOrr]

🙌 Got the lint passing. Thanks @hiranya911

[hiranya911]

LGTM 👍

[hiranya911]

Thanks @ZachOrr. I'm going to merge this now. I will try and get it released next week. We also need to add 1-2 integration test cases to further verify this. I can work on it, or you're also welcome to provide a PR for that.