Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SOL-43240] The queue name ended with the group name #28

Closed
liuz7 opened this issue Sep 29, 2020 · 8 comments · Fixed by #62
Closed

[SOL-43240] The queue name ended with the group name #28

liuz7 opened this issue Sep 29, 2020 · 8 comments · Fixed by #62
Labels
enhancement New feature or request tracked Internally tracked by Solace's internal issue tracking system
Milestone

Comments

@liuz7
Copy link

liuz7 commented Sep 29, 2020

When i add consumer group in configuration, the queue name is always generated with destination name and group name. But our queue name is predefined. We do not need to append the group name. The code for generation of queue name is found below:

private static String getQueueName(String topicName, String groupName,
							SolaceCommonProperties properties,
							boolean isAnonymous, String anonGroupPostfix) {
	String queueName;
	if (isAnonymous) {
		queueName = topicName + QUEUE_NAME_DELIM + JCSMPFactory.onlyInstance().createUniqueName(anonGroupPostfix);
	} else {
		queueName = topicName + QUEUE_NAME_DELIM + groupName;
	}

	return properties.getPrefix() + queueName;
}

please provide a way to specific a queue name without any group name.

@Mrc0113
Copy link
Contributor

Mrc0113 commented Sep 30, 2020

Hi @liuz7,

You're correct, the binder dynamically creates (or expects if you turn auto provisioning off) a queue name that includes the consumer group name on the end. The cloud stream binder follows this naming convention as it's a recommended best practice for a queue in solace to be associated with the consumer/group which consumes from it and this naming format makes that easy to identify.

Does your organization already have a well defined naming convention that you're required to follow? If so, I'm curious if that naming convention includes the consumer group?

@liuz7
Copy link
Author

liuz7 commented Oct 12, 2020

Hi @liuz7,

You're correct, the binder dynamically creates (or expects if you turn auto provisioning off) a queue name that includes the consumer group name on the end. The cloud stream binder follows this naming convention as it's a recommended best practice for a queue in solace to be associated with the consumer/group which consumes from it and this naming format makes that easy to identify.

Does your organization already have a well defined naming convention that you're required to follow? If so, I'm curious if that naming convention includes the consumer group?

We have a pre-defined queue name which does not need to include the group name, would you provide the way for us to use the pre defined queue name?

@Nephery Nephery added the enhancement New feature or request label Oct 29, 2020
@Nephery Nephery added this to the SCSt 3.0.0 milestone Nov 17, 2020
@Nephery
Copy link
Collaborator

Nephery commented Nov 24, 2020

So if I'm understanding this right, the ask here is to allow the ability to have anonymous consumer groups with durable queues and non-random names (or more specifically, have the queue name be equal to the destination name).

If so, then I think we should provide two new consumer config options for this:

Config Option Default Value Description
anonymousGroupDurability false Whether to provision a durable queue for the anonymous group. Note that when this is set to true, then all durable-queue-related config options will also apply to this binding (e.g. provisionDurableQueue).
generateUniqueAnonymousGroupName true Whether to generate a unique name for the anonymous group. Be aware that if this and anonymousGroupDurability are both set to false, then your application may throw errors if it encounters any naming collisions.

And then, we would need to add support for anonymousGroupPostfix to be set to "" (i.e. unset it).

With these changes, you should then be able do your use case by:

  1. Setting generateUniqueAnonymousGroupName to false
  2. Setting anonymousGroupPostfix to ""
  3. Setting anonymousGroupDurability to true

Referencing @liuz7 's code snippet, applying the first two settings will basically make it so that queueName = topicName. And the last setting will make this queue durable instead of temporary.

@Nephery
Copy link
Collaborator

Nephery commented Nov 24, 2020

Ignore my previous comment.

After discussing with @Mrc0113 , we should just introduce a single consumer config option:

Config Option Default Value Description
useGroupNameInQueueName true Whether or not to include the group name in the queue name. If set to false, all consumers of the same destination which have have this set to false will share the same queue regardless of the configured group name.

@Mrc0113 Do we want to introduce another config option to do the same for the error queue?

@Mrc0113
Copy link
Contributor

Mrc0113 commented Nov 25, 2020

@Nephery Sure, for consistency we should probably have the toggle available on the error queue as well. true as default of course

@Mrc0113
Copy link
Contributor

Mrc0113 commented Nov 25, 2020

@Nephery let's update the description to include what changes in the queue name expected as so:

Whether or not to include the group name in the queue name. If set to true the queue name is expected to be {PREFIX}{DESTINATION}.{GROUP}. If set to false, the queue name is expected to be {PREFIX}{DESTINATION}.
Note that if set to false all consumers of the same destination which also have this set to false will consume from the same queue regardless of the configured group name.

@Nephery
Copy link
Collaborator

Nephery commented Nov 25, 2020

Alright, here's the update set of new consumer config options:

Config Option Type Default Value Description
useGroupNameInQueueName boolean true Whether or not to include the group name in the queue name. If set to true, the queue name is expected to be {PREFIX}{DESTINATION}.{GROUP}. If set to false, the queue name is expected to be {PREFIX}{DESTINATION}.
Note that if set to false, all consumers of the same destination which also have this set to false will consume from the same queue regardless of their configured group names.
useGroupNameInErrorQueueName boolean true Whether or not to include the group name in the error queue name. If set to true, the error queue name is expected to be {PREFIX}{DESTINATION}.{GROUP}.error. If set to false, the error queue name is expected to be {PREFIX}{DESTINATION}.error.
Note that if set to false, all consumers of the same destination which also have this set to false will republish failed messages to the same error queue regardless of their configured group names.
errorQueueNameOverride String null A custom error queue name.

For simplicity's sake, I'm assuming we already made the changes to rename the binder DMQ to Error Queue in regard to the wording of useGroupNameInErrorQueueName.

@Nephery Nephery changed the title The queue name ended with the group name [SOL-43240] The queue name ended with the group name Nov 27, 2020
@Nephery Nephery added the tracked Internally tracked by Solace's internal issue tracking system label Nov 27, 2020
@Nephery Nephery linked a pull request Jan 27, 2021 that will close this issue
@Nephery Nephery mentioned this issue Feb 24, 2021
Nephery added a commit that referenced this issue Feb 24, 2021
### Global
* Major version bump to `2.0.0`
* Upgrade to `spring-cloud` `2020.0.1`
* Upgrade to `spring-boot` `2.4.3`
* Upgrade to `sol-jcsmp` `10.10.0`
* Upgrade to `sol-jms` `10.10.0`
* Fix Java 11 build (#38)
* Migrate CI from Travis to Github Actions
* Use Maven Failsafe plugin to run integration tests

### Solace Spring Cloud Stream Binder
* Major version bump to `3.0.0`
* Add Solace Spring Message Headers (#50)
  * Add `SolaceHeaders` and `SolaceBinderHeaders`
  * Bidirectionally map `SolaceHeaders` to JCSMP properties so message handlers can read/write Solace properties
* Renamed "Binder DMQ" to "Error Queue"
* Fix requeuing logic (#13)
  * requeuing is no longer supported for anonymous consumer groups (i.e. temporary queues) since these cannot be rebound.
* Add support for manual acknowledgments (#14)
* Removed the message discard error handling strategy from defined consumer groups. The new default for these will be requeuing.
* Add support for wildcard destinations (#3)
* add consumer config options to omit group name from the consumer group queue or error queue names (#28)
* add `errorQueueNameOverride` consumer config option to override the generated error queue name with a custom config-provided one. (#28)
* Add `headerExclusions` producer config option to exclude headers from published messages
* Add `nonserializableHeaderConvertToString ` producer config option to convert non-serializable headers to strings
* Override the default DMQ eligibility when publishing to be true (#9)
* Remove `solace_raw_message` error-channel message header in favor of `sourceData` header
* Fix JMS interoperability
* Fix `null` payload error handling (#54)
* Fix error handling failures (#36)
* Add `errorMsgDmqEligibility` consumer config option to override failed input messages' DMQ eligibility property when republishing to error queues
* Refactor default generated queue names to be more similar to Solace's shared subscriptions feature
* Fix asynchronous publishing exceptions to be sent to error channels (#34)
* Properly construct the `ErrorMessage` for publisher failures
* Configure client info provider to display Solace SCSt Binder release details
* Reduce warning levels from WARN to INFO when provisioning is disabled or when subscriptions already exist on queues
* Document ACL Profile tips when using error queues (#60)

### Solace Spring Cloud Connector
* Upgrade to `spring-cloud-connectors` `2.2.13.RELEASE` (version managed separately from spring cloud BOM)
@Nephery
Copy link
Collaborator

Nephery commented Feb 25, 2021

Closed with #75

@Nephery Nephery closed this as completed Feb 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request tracked Internally tracked by Solace's internal issue tracking system
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants