Add pubsubTopic field to index

[jm-clius]

Adds pubsubTopic field to Index types.

This is necessary as a single store could serve multiple pubsubTopics. A cursor/index should therefore differentiate between messages that are identical across all fields, but were published on different pubsubTopics.

nim-waku changes corresponding to this RFC change in waku-org/nwaku#864

Is this backwards compatible?

Since adding a field with a new tag value to a protobuf maintains backwards compatibility, there is no need to bump the store protocol id. However, HistoryResponses will now return the pubsubTopic as part of the cursor in PagingInfo. The full PagingInfo, including pubsubTopic, must be provided in subsequent queries to the store node. A future improvement here would be to hash the cursor into a binary blob, so that clients can remain agnostic as to its contents.

[D4nte]

The full PagingInfo, including pubsubTopic, must be provided in subsequent queries to the store node.

This makes it not backward compatible as the client needs to be aware of the pubsubTopic field to ensure it's encoded and send to traverse the pages. So yes, the first query would work but not the following ones.

If we want to make it backward compatible, then I suggest to default to the default pubsub topiic when not present.

[jm-clius]

This makes it not backward compatible as the client needs to be aware of the pubsubTopic field to ensure it's encoded and send to traverse the pages. So yes, the first query would work but not the following ones.

Agreed. Sorry, my description should have been more clear. I meant that protobuf encoding/decoding won't break outright, which I hoped would be enough reason not to introduce another protocol id bump (trying to avoid this as it forces tightly synchronised client/fleet updates). The protocol itself will indeed need to be updated on clients for subsequent queries to work as expected, which makes the change not backwards compatible.

If we want to make it backward compatible, then I suggest to default to the default pubsub topiic when not present.

I'd prefer to avoid this in the long run, as it essentially requires awkwardly injecting data into a deserialised object and could wrongly lead to pubsubTopic being used as if it's a type of optional query criteria. The cursor should in fact be treated more as a blob of data by clients that they must use as-is in subsequent queries than a transparent object with mutable fields (in fact, I hope we can convert it to just a byte string in future). That said, if we'd rather not have a transition period where subsequent queries fail for clients we could temporarily make such a change in nwaku and just remove it after clients have been updated.

[D4nte]

I believe this change is straightforward and can be implemented by store clients easily, even before it's implemented by nim-waku.

Cc @richard-ramos