From 16b6d01ef14dbbdf61e823f519084d3bc060fd1d Mon Sep 17 00:00:00 2001 From: Tomasz Pietrek Date: Wed, 18 Jan 2023 21:16:50 +0100 Subject: [PATCH 1/4] Reorganize ADR to nr 34 Signed-off-by: Tomasz Pietrek --- Readme.,d | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 Readme.,d diff --git a/Readme.,d b/Readme.,d new file mode 100644 index 0000000..e69de29 From 8a969bd9e1e4531f1da8374bef30f9562fc71c1a Mon Sep 17 00:00:00 2001 From: Tomasz Pietrek Date: Thu, 19 Jan 2023 07:36:59 +0100 Subject: [PATCH 2/4] Update the ADR Signed-off-by: Tomasz Pietrek --- adr/ADR-34.md | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 adr/ADR-34.md diff --git a/adr/ADR-34.md b/adr/ADR-34.md new file mode 100644 index 0000000..6fbf2a9 --- /dev/null +++ b/adr/ADR-34.md @@ -0,0 +1,55 @@ +# Title JetStream Consumers Multiple Filters + +| Metadata | Value | +|----------|---------------------------| +| Date | 2023-01-18 | +| Author | @Jarema | +| Status | Approved | +| Tags | jetstream, client, server | + +## Context and Problem Statement + +Initially, JetStream Consumers could have only one Filter Subject. +As the number of feature requests to specify multiple subjects increased, this feature was added. +That could also reduce the number of Consumers in general. + +## Context + +Server PR: https://github.com/nats-io/nats-server/pull/3500 + +## Design + +### Client side considerations + +To implement the feature without any breaking changes, a new field should be added to Consumer config, both on the server and the clients. + +1. The new field is: +`FilterSubjects []string json:"filter_subjects"` + +2. Subjects can't overlap each other and have to fit the interest of the Stream. +In case of overlapping subjects, error (10136) will be returned. + +3. Only one of `FilterSubject` or `FilterSubjects` can be passed. Passing both results in an error (10134) + +4. Until future improvements, only old JS API for consumer creation can be used (the one without `FilterSubject`) in consumer create request. Using new API will yield an error (10135). + +5. Each client, to support this feature, needs to add a new field that Marshals into a json array of strings. + +6. To ensure compatibility with old servers that are not aware of `filter_subjects` field, client's should check the returned info (from update or create) if the filters are set up properly. + +**Example** +```json +{ + "durable_name": "consumer", + "filter_subjects": ["events", "data"] +} +``` +6. Client does not have to check if both, single and multiple filters were passed, as server will validate it. +Client should add new errors to return them in client language idiomatic fashion. + +### Server side + +To make this change possible and reasonable peformant, server will have a buffer of first message for each subject filtered and will deliver them in order. After delivering message for a subject, buffer for that subject will be filled again, resulting in close to no overhead after the initial buffer fill. + +This can be optimized but will not affect API. + From d97acdc2601c1a56456ce2f171696b9c2248eea8 Mon Sep 17 00:00:00 2001 From: Tomasz Pietrek Date: Sat, 21 Jan 2023 12:20:03 +0100 Subject: [PATCH 3/4] Update Readme.md --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index c9d9965..e2c33c9 100644 --- a/README.md +++ b/README.md @@ -97,6 +97,7 @@ This repo is used to capture architectural and design decisions as a reference o |[ADR-30](adr/ADR-30.md)|server|Subject Transform| |[ADR-31](adr/ADR-31.md)|jetstream, client, server|JetStream Direct Get| |[ADR-33](adr/ADR-33.md)|jetstream, client, server|Metadata for Stream and Consumer| +|[ADR-34](adr/ADR-34.md)|jetstream, client, server|Title JetStream Consumers Multiple Filters| ## When to write an ADR From 4f975efeff3a98f7ed292eb61f68c458c495e570 Mon Sep 17 00:00:00 2001 From: Tomasz Pietrek Date: Wed, 25 Jan 2023 09:48:40 +0100 Subject: [PATCH 4/4] Finalize ADR Signed-off-by: Tomasz Pietrek --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index e2c33c9..49e75f7 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,7 @@ This repo is used to capture architectural and design decisions as a reference o |[ADR-31](adr/ADR-31.md)|jetstream, client, server|JetStream Direct Get| |[ADR-32](adr/ADR-32.md)|client|Service API| |[ADR-33](adr/ADR-33.md)|jetstream, client, server|Metadata for Stream and Consumer| +|[ADR-34](adr/ADR-34.md)|jetstream, client, server|Title JetStream Consumers Multiple Filters| ## Jetstream @@ -52,6 +53,7 @@ This repo is used to capture architectural and design decisions as a reference o |[ADR-28](adr/ADR-28.md)|jetstream, server|JetStream RePublish| |[ADR-31](adr/ADR-31.md)|jetstream, client, server|JetStream Direct Get| |[ADR-33](adr/ADR-33.md)|jetstream, client, server|Metadata for Stream and Consumer| +|[ADR-34](adr/ADR-34.md)|jetstream, client, server|Title JetStream Consumers Multiple Filters| ## Kv