Make S3 configuration for large messages optional

[VladimirMakaev]

In our setup we don't plan to have large messages that wouldn't fit in the queue. Therefore we don't want to create redundant infrastructure (S3 buckets) which serves no purpose.

Can you make this feature optional and throw an exception in case we try to insert a large message into SQS?

[mauroservienti]

@VladimirMakaev the feature is already optional. The S3 bucket configuration is optional, see:
https://docs.particular.net/transports/sqs/configuration-options#s3bucketforlargemessages

What type of issue are you facing?

[VladimirMakaev]

If I don't specify S3 configuration I'm getting

Unhandled Exception: Amazon.Runtime.AmazonClientException: No RegionEndpoint or ServiceURL configured

To get rid of the error message I have to do:

transport.S3("do-not-use", "do-not-use");

Which I think is ambiguous. And in the case a large message is attempted to be sent I'd like to see an exception telling me that "I should configure S3 if I want to send large messages"

[danielmarbach]

@VladimirMakaev

Right now the S3Client is created early even though we might not need it so we could definitely think about creating it only when needed at a later point in time.

But to better understand the issue you are seeing I would like to know where you configure the region for the SQS client? Do you use Environment variables? Or do you provide a custom SQS client factory in code? If you provide a custom factory in code could you also override the custom S3 factory to assign the same credentials and remove the S3 bucket configuration above and see if it fixes it? If you are using Environment variables can you make sure you have AWS_REGION set because then it should just work?

[VladimirMakaev]

I do provide a custom factory. To provide a custom S3 client factory I need to do transport.S3 which requires me to provide name of the bucket and keyPrefix which I don't want to know about at that stage.

[danielmarbach]

Yeah totally understandable. That is inconvenient and something we can fix as part of a next maintenance release. For now a "workaround" for you so that you can still get the exception when you send large messages and not need to provide the bucket names you can set the AWS_REGION environment variable. Then it should start.

[lalitkale]

Due to this defect, we wasted couple of days. It is also, not helpful when exception does not exactly point out that it is NServicebus issue or its AWS SDK issue. Atleast, exception should made clear that what we aer missing in configuration.

[boblangley]

@lalitkale yes, we agree. The exceptions are not very helpful. I believe this is what #187 is raising as well. I marked that as candidate-for-next-release to try to tackle when we do our next maintenance release.

[lalitkale]

@boblangley Any ETA on next release?

[danielmarbach]

@lalitkale thanks for your patience. We are working on a maintenance release that should roughly go out at the end of next week or beginning of the week after next week. I'll keep you posted here

[danielmarbach]

We've merged a PR that makes the S3 configuration optional. Those changes will go out with 4.3.0