diff --git a/docs/building-a-template.md b/docs/building-a-template.md index 7adce05f..da46462b 100644 --- a/docs/building-a-template.md +++ b/docs/building-a-template.md @@ -87,6 +87,7 @@ When creating your watchbot stacks with the `watchbot.template()` method, you no **alarmPeriods** | Use this parameter to control the duration that the SQS queue must be over the message threshold before triggering an alarm. You specify the number of 5-minute periods before an alarm is triggered. The default is 24 periods, or 2 hours. This parameter can be provided as either a number or a reference, i.e. `{"Ref": "..."}`. | String/Ref | No | 24 **alarmThreshold** | Watchbot creates a CloudWatch alarm that will go off when there have been too many messages in SQS for a certain period of time. Use this parameter to adjust the Threshold number of messages to trigger the alarm. This parameter can be provided as either a number or a reference, i.e. `{"Ref": "..."}`. | Number/Ref | No | 40 **errorThreshold** | Watchbot creates a CloudWatch alarm that will fire if there have been more than this number of failed worker invocations in a 60 second period. This parameter can be provided as either a number or a reference, i.e. `{"Ref": "..."}`. | Number/Ref | No | 10 +**deadletterThreshold** | Use this parameter to control the number of times a message is delivered to the source queue before being moved to the dead-letter queue. This parameter can be provided as either a number or a reference, i.e. `{"Ref": "..."}`. | Number/Ref | No | 10 **dashboard** | Watchbot creates a Cloudwatch Dashboard called `-`. If running in cn-north-1, this may need to be disabled | Boolean | No | `true` **fifo** | Whether you want Watchbot's SQS queue to be first-in-first-out (FIFO). By default, Watchbot creates a standard SQS queue, in which the order of jobs is not guaranteed to match the order of messages. If your program requires more precise ordering and the limitations of a FIFO queue will be acceptable, set this option to `true`. Learn more in ["Using a FIFO queue"](./using-a-fifo-queue.md) | Boolean | No | `false` diff --git a/docs/using-a-fifo-queue.md b/docs/using-a-fifo-queue.md index a1956e02..01ad250c 100644 --- a/docs/using-a-fifo-queue.md +++ b/docs/using-a-fifo-queue.md @@ -4,14 +4,24 @@ By default, Watchbot creates a standard SQS queue, in which the order of jobs is To use a FIFO queue, set the template option `fifo: true`. -A FIFO queue behaves differently and has some limitations, so you should read the AWS documentation and the details below if you are considering the switch. +A FIFO queue behaves differently and has some limitations, so you should read [the AWS documentation](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html) and the details below if you are considering the switch. ### Triggering FIFO workers -With a standard queue, you trigger a Watchbot worker by sending an SNS message to the watcher's topic. With a FIFO queue, you will send a message directly to the watcher's FIFO queue, instead. Make sure you have read about the AWS documentation about the expected for these messages (for example, you need to include a `MessageGroupId`). +With a standard queue, you trigger a Watchbot worker by sending an SNS message to the watcher's topic. With a FIFO queue, you send a message directly to the watcher's FIFO queue, instead. Make sure you have read [the AWS documentation](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/FIFO-queues.html) about the expected format of FIFO queue messages (for example, you need to include a `MessageGroupId`). ### Scaling FIFO workers [Scaling behavior](./scaling-in-watchbot.md) in Watchbot is based on the total number of messages in the queue. If there are 50 messages, concurrent processing scales up to 50 (or the configured `maxSize`, if lower). In a FIFO queue, this scaling will not always make sense. For example, if your queue includes 50 messages but they only span 2 `MessageGroupId`s, then you will only be able to process 2 messages at a time and 48 worker machines will sit idle. For this reason, it's important that you plan for the number of `MessageGroupId`s you will use and set the template option `maxSize` accordingly. If you are only going to use 2 `MessageGroupId`s, you should set `maxSize: 2`. + +### Dead-letter queues and FIFO + +[The AWS documentation for dead-letter queues](https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html) suggests the following: + +> Don’t use a dead-letter queue with a FIFO queue if you don’t want to break the exact order of messages or operations. + +If exact ordering is absolutely necessary for your program, you may prefer that the whole system stall if one message fails, instead of dropping the failed message into a dead-letter queue and moving on to the next one. + +Currently there is no way to turn off Watchbot's dead-letter queue. If you need this feature, please submit an issue or pull request.