Skip to content

Commit

Permalink
Document that we are improving the SQS Notifications interface (#7656)
Browse files Browse the repository at this point in the history
  • Loading branch information
quodlibetor authored Aug 2, 2021
1 parent 7711faf commit 12c104c
Showing 1 changed file with 9 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -75,12 +75,15 @@
queue with **DISCOVER OBJECTS USING SQS NOTIFICATIONS `'queue-name'`**.

Materialize deletes SQS messages as soon as they are ingested. This means that the same SQS queue
cannot be used for multiple sources. Since Materialize treats unmaterialized sources with multiple
downstream views as separate sources, `SQS NOTIFICATIONS` should not be used with unmaterialized
sources. If you would like to have multiple sources listening to notifications from the same bucket
you must [configure an SNS topic][sns] as an intermediary, with multiple SQS queues subscribed to
it. Note that SQS queues subscribed to SNS topics intended for Materialize must be configured to
use [raw message delivery][raw-delivery].
cannot be used for multiple sources. If you would like to have multiple sources listening to
notifications from the same bucket you must [configure an SNS topic][sns] as an intermediary, with
multiple SQS queues subscribed to it, one SQS queue per source. Note that SQS queues subscribed to
SNS topics intended for Materialize must be configured to use [raw message delivery][raw-delivery].

Since Materialize treats unmaterialized sources with multiple downstream views as separate sources,
`SQS NOTIFICATIONS` should not be used with unmaterialized sources. This behavior of unmaterialized
sources is considered a bug ([#7423](https://github.com/MaterializeInc/materialize/issues/7423)),
and will be improved in a future release.

[notification-aws]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/NotificationHowTo.html
[notification-tutorial]: https://docs.aws.amazon.com/AmazonS3/latest/userguide/ways-to-add-notification-config-to-bucket.html
Expand Down

0 comments on commit 12c104c

Please sign in to comment.