message unreceivable without first decoding from base64

[nateq314]

Was following the setup instructions per the readme and just spent a really long time debugging an issue that was causing the following

{
  "data": {
    "somethingChanged": {
      "message": null
    }
  }
}

to be received each time. (I modified it a tiny bit and replace "id" with "message".)

Schema was:

  type Result {
    message: String
  }

  type Subscription {
    somethingChanged: Result
  }

Resolver was:

  somethingChanged: {
    subscribe: () => pubsub.asyncIterator(SOMETHING_CHANGED_TOPIC)
  }

Basically everything almost exactly per the instructions, but I kept getting null. I tried both 1) directly publishing a message via the Pub/Sub GUI on Google Cloud Console, and
2) setting up a dummy mutation as follows, and calling it:

  foo() {
    pubsub.publish(SOMETHING_CHANGED_TOPIC, { message: "hello" });
    return {
      message: "hello"
    };
  },

Both kept resulting in null.

What finally fixed it was adding a resolve() method to the somethingChanged subscription which decodes from base64, as follows:

  somethingChanged: {
    resolve: (payload: any) => {
      const utf8 = Buffer.from(payload.data, "base64").toString("utf-8");
      return JSON.parse(utf8);
    },
    subscribe: () => pubsub.asyncIterator(SOMETHING_CHANGED_TOPIC)
  }

Or putting the same into commonMessageHandler:

const commonMessageHandler = (message: any) => {
  try {
    const utf8 = Buffer.from(message.data, "base64").toString("utf-8");
    return JSON.parse(utf8);
  } catch {
    return {};
  }
};

export const pubsub = new GooglePubSub(
  {
    projectId: "[my-project-id]",
    credentials
  },
  undefined,
  commonMessageHandler
);

with resolve() just passing on the payload as is (doesn't work without this):

  somethingChanged: {
    resolve: (payload: any) => payload,
    subscribe: () => pubsub.asyncIterator(SOMETHING_CHANGED_TOPIC)
  }

Google's own docs clearly state that base64 is necessary (https://cloud.google.com/pubsub/docs/publisher). So given that this is a Google-cloud-specific package, isn't this the kind of thing that ought to be handled behind the scenes by the package? Am I doing something wrong? Has anybody else run into this?

[jonas-app]

@nateq314
In our case (mostly string only data) we use a commonMessageHandler and just call toString on the data buffer.

function commonMessageHandler ({attributes = {}, data = ''}) {
  return {
    ...attributes,
    text: data.toString()
  }
}

If you publish non string data it gets stringified and you have to parse the received message data as you did above.

graphql-google-pubsub/src/google-pubsub.ts

Line 28 in 7e70c8f

data = JSON.stringify(data);

With the commonMessageHandler you are quite flexible with what you can do with the received message.

Let me know if you have any non breaking suggestions which would improve this handling.
Also do not hesitate to point out improvements to the readme or even open a PR.

[jonas-app]

@nateq314
You can also have a look at the following test to see the potential of the flexible commonMessageHandler and why I have decided against putting the JSON.parse() somewhere directly into this package.

graphql-google-pubsub/src/test/tests.ts

Line 223 in 7e70c8f

it('can use custom message handler', done => {

Also there could be anything else in the buffer apart from a simple string.

[isdelrey]

I had the same issue. Maybe this explanation could be added to the README because by just following the instructions there, you only get that null.

[jonas-app]

@nateq314 @ivosequeros thx for the feedback. Will move this information over to the readme as soon as I find time.