PubSub http connector PubSubMessage data should be an Option

[jacobg]

Versions used

Akka version: 2.5.23

Expected Behavior

Start up stream reading from PubSub subscription, and successfully read message that has no data. The PubSubMessage data attribute should have value of None.

Actual Behavior

Exception is thrown:

10:05:41.703 [EventStreamSpec-akka.actor.default-dispatcher-2] WARN akka.stream.Materializer - [before-map] Upstream failed, cause: DeserializationException: Object is missing required member 'data'

And even if a recover stage is added to prevent stream closing, still we don't have the message ackId in order to move on. So the message will keep repeating!

Please provide a description of what actually happens, working from the same starting point.

The PubSubMessage data attribute is of type String, whereas it should be of type Option[String] so that it can read in a null or missing json value. Google documents that it is supported to not specify data as long as there are attributes:

Note that the message payload must not be empty; it must contain either a non-empty data field, or at least one attribute.

https://cloud.google.com/pubsub/docs/publisher#publish

Indeed, using the Pub/Sub Java client, a message can be published without data.

Relevant logs

If you identified a section in your logs that explains the bug or might be important to understand it, please add it.

Reproducible Test Case

Please provide a PR with a failing test.

If the issue is more complex or requires configuration, please provide a link to a project that reproduces the issue.

[jacobg]

@stephennancekivell @2m @ennru

[stephennancekivell]

Oh my. I didn't know of that use case. Sounds like a good first issue for someone.

It's a shame this change will break compatibility with existing code, but I don't see anyway around it.

[ennru]

Yeah, makes sense. We expect to do an API-breaking 1.1 release in September, probably.

[ennru]

Related to #1786

[2m]

In the light of #1786 (comment) we could make the data contain an empty string, if the received message does not contain any data.

[ennru]

Would be great to sort this out for Alpakka 2.0.

[ennru]

This was fixed with #1881 in Alpakka 2.0.0-M1.