Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid max queue name length, hide those implementieren detail to the user #88

Closed
wants to merge 1 commit into from
Closed

Avoid max queue name length, hide those implementieren detail to the user #88

wants to merge 1 commit into from

Conversation

GreenRover
Copy link
Contributor

I user entered a to long destination, he ends up with:

com.solacesystems.jcsmp.JCSMPErrorResponseException: 400: Invalid Queue Name
	at com.solacesystems.jcsmp.impl.flow.BindRequestTask.execute(BindRequestTask.java:204) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.impl.flow.SubFlowManagerImpl.handleAssuredCtrlMessage(SubFlowManagerImpl.java:571) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.handleAssuredCtrlMsg(TcpClientChannel.java:1656) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.handleMessage(TcpClientChannel.java:1621) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SubscriberMessageReader.processRead(SubscriberMessageReader.java:98) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SubscriberMessageReader.read(SubscriberMessageReader.java:138) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.read(SimpleSmfClient.java:1168) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SyncEventDispatcherReactor.processReactorChannels(SyncEventDispatcherReactor.java:206) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SyncEventDispatcherReactor.eventLoop(SyncEventDispatcherReactor.java:157) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SyncEventDispatcherReactor$SEDReactorThread.run(SyncEventDispatcherReactor.java:338) ~[sol-jcsmp-10.10.0.jar:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]
2021-05-31T21:31:46.374+0200 ERROR Failed to create consumer binding; retrying in 30 seconds
org.springframework.cloud.stream.provisioning.ProvisioningException: Failed to connect test consumer flow to queue scst/an/785c9fec-2ebc-4d13-920c-6c9e67883ff1/plain/moren/then/144/char/long/destination/provided/with/a/very/very/big/smile/just/for/you/and/even/more/and/more/and/more/chars/to/be/ultra/mega/long; nested exception is ((Client name: pub_sub_receiving_K65710_400367be-a16f-42a9-a24f-590c30b180c7   Local addr: 192.168.50.113 Local port: 62556   Remote addr: solace.thehennings.ch  Remote port: 55555) - )  com.solacesystems.jcsmp.JCSMPErrorResponseException: 400: Invalid Queue Name
	at com.solace.spring.cloud.stream.binder.provisioning.SolaceQueueProvisioner.provisionQueue(SolaceQueueProvisioner.java:184) ~[spring-cloud-stream-binder-solace-core-3.1.0.jar:?]
	at com.solace.spring.cloud.stream.binder.provisioning.SolaceQueueProvisioner.provisionQueue(SolaceQueueProvisioner.java:140) ~[spring-cloud-stream-binder-solace-core-3.1.0.jar:?]
	at com.solace.spring.cloud.stream.binder.provisioning.SolaceQueueProvisioner.provisionConsumerDestination(SolaceQueueProvisioner.java:124) ~[spring-cloud-stream-binder-solace-core-3.1.0.jar:?]
	at com.solace.spring.cloud.stream.binder.provisioning.SolaceQueueProvisioner.provisionConsumerDestination(SolaceQueueProvisioner.java:33) ~[spring-cloud-stream-binder-solace-core-3.1.0.jar:?]
	at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:403) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindConsumer(AbstractMessageChannelBinder.java:91) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binder.AbstractBinder.bindConsumer(AbstractBinder.java:143) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binding.BindingService.doBindConsumer(BindingService.java:177) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binding.BindingService.bindConsumer(BindingService.java:134) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binding.AbstractBindableProxyFactory.createAndBindInputs(AbstractBindableProxyFactory.java:118) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binding.InputBindingLifecycle.doStartWithBindable(InputBindingLifecycle.java:58) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at java.util.LinkedHashMap$LinkedValues.forEach(LinkedHashMap.java:608) ~[?:?]
	at org.springframework.cloud.stream.binding.AbstractBindingLifecycle.start(AbstractBindingLifecycle.java:57) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.cloud.stream.binding.InputBindingLifecycle.start(InputBindingLifecycle.java:34) ~[spring-cloud-stream-3.1.3.jar:3.1.3]
	at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:178) ~[spring-context-5.3.7.jar:5.3.7]
	at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:54) ~[spring-context-5.3.7.jar:5.3.7]
	at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:356) ~[spring-context-5.3.7.jar:5.3.7]
	at java.lang.Iterable.forEach(Iterable.java:75) [?:?]
	at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:155) [spring-context-5.3.7.jar:5.3.7]
	at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123) [spring-context-5.3.7.jar:5.3.7]
	at org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:935) [spring-context-5.3.7.jar:5.3.7]
	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:586) [spring-context-5.3.7.jar:5.3.7]
	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:144) [spring-boot-2.4.6.jar:2.4.6]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:771) [spring-boot-2.4.6.jar:2.4.6]
	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:763) [spring-boot-2.4.6.jar:2.4.6]
	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:438) [spring-boot-2.4.6.jar:2.4.6]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:339) [spring-boot-2.4.6.jar:2.4.6]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1329) [spring-boot-2.4.6.jar:2.4.6]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1318) [spring-boot-2.4.6.jar:2.4.6]
	at ch.sbb.tms.springcloudstream.examples.pubsubreceiving.PubSubReceivingApplication.main(PubSubReceivingApplication.java:9) [classes/:?]
Caused by: com.solacesystems.jcsmp.JCSMPErrorResponseException: 400: Invalid Queue Name
	at com.solacesystems.jcsmp.impl.flow.BindRequestTask.execute(BindRequestTask.java:204) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.impl.flow.SubFlowManagerImpl.handleAssuredCtrlMessage(SubFlowManagerImpl.java:571) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.handleAssuredCtrlMsg(TcpClientChannel.java:1656) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.impl.TcpClientChannel.handleMessage(TcpClientChannel.java:1621) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SubscriberMessageReader.processRead(SubscriberMessageReader.java:98) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SubscriberMessageReader.read(SubscriberMessageReader.java:138) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.smf.SimpleSmfClient.read(SimpleSmfClient.java:1168) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SyncEventDispatcherReactor.processReactorChannels(SyncEventDispatcherReactor.java:206) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SyncEventDispatcherReactor.eventLoop(SyncEventDispatcherReactor.java:157) ~[sol-jcsmp-10.10.0.jar:?]
	at com.solacesystems.jcsmp.protocol.nio.impl.SyncEventDispatcherReactor$SEDReactorThread.run(SyncEventDispatcherReactor.java:338) ~[sol-jcsmp-10.10.0.jar:?]
	at java.lang.Thread.run(Thread.java:834) ~[?:?]

Those implementation details should be hide from the user.
Please see solace internal ticket #51636 for further discussion.

@Nephery
Copy link
Collaborator

Nephery commented Oct 25, 2021

After discussing with our architects, we decided that the binder shouldn't trim queue names. This will introduce a pretty high risk for queue name collisions when destinations are very long.

Alternatively, what if we introduced a toggle-able feature that allowed for hashing of the destination in the generated queue name? That way you can preserve the uniqueness of generated queue names while both using really long destinations as well as have support for characters that are supported in topic subscription names but not in queue names ('<>*?&;).

So when this feature is enabled, the generated queue name will look something like:

<prefix>/<familiarity-modifier>/<group>/hash/<fixed-length-hashed-destination>

@Nephery Nephery closed this Oct 25, 2021
@Nephery
Copy link
Collaborator

Nephery commented Nov 10, 2021

Ignore what I said in my previous comment. We'll be adding a config option which lets you define your queue names (or modify the existing generated ones) using SpEL expressions. Within which, you can trim queue names if you want.

e.g. for this particular PR, you could modify this config to be:

spring.cloud.stream.solace.bindings.<binding-name>.consumer.queueNameExpression=T(org.apache.commons.lang3.StringUtils).left(<default-queue-name-expression>, group matches '\\s*' ? 143 : 200)

Nephery pushed a commit that referenced this pull request Mar 4, 2022
* add config options to change the queue name format:
  * Add `queueNameExpression` and `errorQueueNameExpression` consumer config options
  * Add `queueNameExpression` and `queueNameExpressionsForRequiredGroups` producer config options
  * Deprecate `queueNamePrefix`, `useGroupNameInQueueName`, `useFamiliarityInQueueName`, `useDestinationEncodingInQueueName`, `errorQueueNameOverride`, and `useGroupNameInErrorQueueName` binding config options
  * Fixes #88
@Nephery Nephery mentioned this pull request Mar 4, 2022
PhilippeKhalife added a commit that referenced this pull request Mar 11, 2022
* Add is-reply message header (#97)

* Add `solace_isReply` message header

* Creating configuration metadata for bindings properties (#98)

* Handle consuming null payloads (#112)

* Add `solace_scst_nullPayload` message header
* Convert incoming null payloads to empty equivalents

* add config option to disable adding destination subscription on queue (#108)

* add `addDestinationAsSubscriptionToQueue` binding config option
* deprecate `isProvisionSubscriptionsToDurableQueue` in favor of `addDestinationAsSubscriptionToQueue`

Co-authored-by: Jeffrey D <11084623+Nephery@users.noreply.github.com>

* Upgrade to Spring cloud 2021.0.0 and Junit 5 (#118)

# Product Changes
* Upgrade Spring Boot to `2.6.2`
* Upgrade Spring Cloud to `2021.0.0`
* Upgrade sol-jcsmp to `10.13.0`

# Test Changes
* Upgrade sol-jms to `10.13.0`
* Upgrade solace-integration-test-support to `0.8.0`
* Upgrade all binder tests to JUnit 5
  * Add `PubSubPlusExtension` into integration tests for automatic provisioning of PubSub+ in Docker and extension-managed JCSMP resources (e.g. managed `JCSMPSession` and `Queue`).
    * Docker provisioning is disabled if Spring properties are defined to use an externally provisioned broker.
  * Usage of Mockito, SoftAssertions, & ExecutorService were updated to use extensions for injection and auto-cleanup (or auto-calling `assertAll()` in the case of SoftAssertions).
* Remove Maven `it` profile
  * Integration tests will always run by default.
* Add parallel test execution and enable it by default in **Maven**
  * Is disabled by default for easy debugging when test are directly ran from an IDE (not through Maven).
* Update `Test` Github Action workflow to skip duplicate runs, cache maven dependencies, and report unit test results.

* fix PubSubPlusExtension integration (#119)

* upgrade solace-integration-test-support to 0.9.0
* fix `SpringCloudStreamExtension` to use static accessors of `PubSubPlusExtension` resources

* Replace the use of Pivotal Cloud Foundry by VMware Tanzu in documentation (#114)

* SOL-46352: Fix duplicate logs by registering a single RetryListener (#116)

* Fix duplicate logs by registering a single RetryListener
  * Fixes #77

* update sonatype urls to s01 (#121)

Co-authored-by: skgaddam <ShivaKumar.Gaddam@solace.com>

* Delivery Count (#122)

* Add solace_deliveryCount Spring message header
  * This is a controlled availability feature

* upgrade to spring-cloud@2021.0.1 & spring-boot@2.6.4 (#126)

* SOL-57956: Removing dependency on spring-cloud-connector (#113)

Fixes #15

* Sol 46284 Queue Name Expression (#120)

* add config options to change the queue name format:
  * Add `queueNameExpression` and `errorQueueNameExpression` consumer config options
  * Add `queueNameExpression` and `queueNameExpressionsForRequiredGroups` producer config options
  * Deprecate `queueNamePrefix`, `useGroupNameInQueueName`, `useFamiliarityInQueueName`, `useDestinationEncodingInQueueName`, `errorQueueNameOverride`, and `useGroupNameInErrorQueueName` binding config options
  * Fixes #88

* Sol 58058 health actuator (#125)

* Add support for health indicator
* Add new `RECONNECTING` health indicator status

Co-authored-by: E521877 <nathanael.weber@sbb.ch>
Co-authored-by: Jeffrey Douangpaseuth <11084623+Nephery@users.noreply.github.com>

* SOL-58059 Consumer Pause/Resume (#123)

* Add support for consumer pause/resume

Co-authored-by: Jeffrey Douangpaseuth <11084623+Nephery@users.noreply.github.com>

* SOL-43137 Batch Consumer Support (#124)

* Add support for batch consumers
  * Add `batchMaxSize` and `batchTimeout` consumer config options
  * Add `solace_scst_batchedHeaders` message header
  * closes #22

* SOL-49780: SCSt README Cleanup (#127)

* Expand table of contents to 3 levels
* Rewrite overview to be more clear
* Try to make it more apparent that Solace session configuration is handled by the Solace Java Spring Boot starter (a completely different project)
* Remove references to obsolete `prefix` consumer/producer config options. The `prefix` option was removed in `3.0.0`.

* prepare for 2.3.0 release (#128)

* add dependabot (#130)

closes #78

* Upgrade Solace APIs to 10.13.1 & Log4j2 to 2.17.2 (#131)

* Upgrade sol-jcsmp to `10.13.1`
* Upgrade sol-jms to `10.13.1`
* Upgrade Log4j2 to `2.17.2`

* upgrade solace-spring-boot to 1.2.0 (#129)

* upgrade to solace-spring-boot@1.2.1 & solace-services-info@0.4.4

Co-authored-by: carolmorneau <carol.morneau@solace.com>
Co-authored-by: PhilippeKhalife <philippe.khalife@solace.com>
Co-authored-by: skgaddam <ShivaKumar.Gaddam@solace.com>
Co-authored-by: E521877 <nathanael.weber@sbb.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants