Skip to content

Commit

Permalink
NO-JIRA clarify LVQ docs on clustering
Browse files Browse the repository at this point in the history
  • Loading branch information
@jbertram
jbertram committed Feb 21, 2022
1 parent 178b6ce commit 3be4dd6
Showing 1 changed file with 61 additions and 28 deletions.
89 changes: 61 additions & 28 deletions docs/user-manual/en/last-value-queues.md
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,20 @@
# Last-Value Queues

Last-Value queues are special queues which discard any messages when a
newer message with the same value for a well-defined Last-Value property
is put in the queue. In other words, a Last-Value queue only retains the
last value.
Last-Value queues are special queues which discard any messages when a newer
message with the same value for a well-defined Last-Value property is put in
the queue. In other words, a Last-Value queue only retains the last value.

A typical example for Last-Value queue is for stock prices, where you
are only interested by the latest value for a particular stock.
A typical example for Last-Value queue is for stock prices, where you are only
interested by the latest value for a particular stock.

Messages sent to an Last-Value queue without the specified property will be delivered as normal and will never be "replaced".
Messages sent to an Last-Value queue without the specified property will be
delivered as normal and will never be "replaced".

## Configuration

#### Last Value Key Configuration
Last-Value queues can be statically configured in broker.xml via the `last-value-key`
Last-Value queues can be statically configured in broker.xml via the
`last-value-key`

```xml
<address name="foo.bar">
Expand All @@ -34,22 +35,21 @@ Queue queue = session.createQueue("my.destination.name?last-value-key=reuters_co
Topic topic = session.createTopic("my.destination.name?last-value-key=reuters_code");
```

Address wildcards can be used to configure Last-Value queues
for a set of addresses (see [here](wildcard-syntax.md)).
Address wildcards can be used to configure Last-Value queues for a set of
addresses (see [here](wildcard-syntax.md)).

```xml
<address-setting match="lastValueQueue">
<default-last-value-key>reuters_code</default-last-value-key>
</address-setting>
```

By default, `default-last-value-key` is null.

By default, `default-last-value-key` is `null`.

#### Legacy Last Value Configuration

Last-Value queues can also just be configured via the `last-value` boolean property, doing so it will default the last-value-key to `"_AMQ_LVQ_NAME"`.

Last-Value queues can also just be configured via the `last-value` boolean
property, doing so it will default the last-value-key to `_AMQ_LVQ_NAME`.

```xml
<address name="foo.bar">
Expand Down Expand Up @@ -84,15 +84,14 @@ By default, `default-last-value-queue` is false.
Note that `address-setting` `last-value-queue` config is deprecated, please use
`default-last-value-queue` instead.



## Last-Value Property

The property name used to identify the last value is configurable
at the queue level mentioned above.

If using the legacy setting to configure an LVQ then the default property `"_AMQ_LVQ_NAME"` is used
(or the constant `Message.HDR_LAST_VALUE_NAME` from the Core API).
If using the legacy setting to configure an LVQ then the default property
`"_AMQ_LVQ_NAME"` is used (or the constant `Message.HDR_LAST_VALUE_NAME` from
the Core API).

For example, using the sample configuration

Expand Down Expand Up @@ -127,16 +126,23 @@ TextMessage messageReceived = (TextMessage)messageConsumer.receive(5000);
System.out.format("Received message: %s\n", messageReceived.getText());
```


## Forcing all consumers to be non-destructive
When a consumer attaches to a queue, the normal behaviour is that messages are sent to that consumer are acquired exclusively by that consumer, and when the consumer acknowledges them, the messages are removed from the queue.

Another common pattern is to have queue "browsers" which send all messages to the browser, but do not prevent other consumers from receiving the messages, and do not remove them from the queue when the browser is done with them. Such a browser is an instance of a "non-destructive" consumer.
When a consumer attaches to a queue, the normal behaviour is that messages are
sent to that consumer are acquired exclusively by that consumer, and when the
consumer acknowledges them, the messages are removed from the queue.

If every consumer on a queue is non destructive then we can obtain some interesting behaviours. In the case of a LVQ then the queue will always contain the most up to date value for every key.
Another common pattern is to have queue "browsers" which send all messages to
the browser, but do not prevent other consumers from receiving the messages,
and do not remove them from the queue when the browser is done with them. Such
a browser is an instance of a "non-destructive" consumer.

A queue can be created to enforce all consumers are non-destructive for last value queue. This can be achieved using the following queue configuration:
If every consumer on a queue is non destructive then we can obtain some
interesting behaviours. In the case of a LVQ then the queue will always contain
the most up to date value for every key.

A queue can be created to enforce all consumers are non-destructive for last
value queue. This can be achieved using the following queue configuration:

```xml
<address name="foo.bar">
Expand Down Expand Up @@ -164,15 +170,42 @@ Also the default for all queues under and address can be defaulted using the
</address-setting>
```

By default, `default-non-destructive` is false.
By default, `default-non-destructive` is `false`.

#### Bounding size using `expiry-delay`

For queues other than LVQs, having only non-destructive consumers could mean
that messages would never get deleted, leaving the queue to grow unconstrained.
To prevent this you can use the ability to set a default `expiry-delay`.

See [expiry-delay](message-expiry.md#configuring-expiry-delay) for more details
on this.

## Clustering

The fundamental ideas behind last-value queues and clustering are at odds with
each other.

#### Bounding size using expiry-delay
For queues other than LVQs, having only non-destructive consumers could mean that messages would never get deleted, leaving the queue to grow unconstrainedly. To prevent this you can use the ability to set a default `expiry-delay`.
Clustering was designed as a way to increase message throughput through
horizontal scaling. The messages in a clustered queue can be spread across
_all_ nodes in the cluster. This allows clients to be distributed across the
cluster to leverage the computing resources all the nodes rather than being
bottlenecked on a single node.

See [expiry-delay](message-expiry.md#configuring-expiry-delay) for more details on this.

However, if you wanted to use a last-value queue in a cluster then in order
to enforce last-value semantics all messages would be required to go to a
queue on a _single_ node. This would effectively _nullify_ the benefits of
clustering. Also, the arrival of messages on and and redistribution of
those messages from nodes other than the node where the last-value semantics
would be enforced would almost certainly impact which message is considered
"last."

For these reasons last-value queues are not supported in a traditional cluster.
However, it would be possible to use a [broker balancer](broker-balancers.md)
in front of a cluster (or even a set of non-clustered brokers) to ensure all
clients which need to use the same last-value queue are directed to the same
node. See the [broker balancer chapter](broker-balancers.md) for more details
on configuration, etc.

## Example

Expand Down

0 comments on commit 3be4dd6

Please sign in to comment.