Allow local message storage when ASB is down #22
Comments
|
Is this store-and-forward on the cloud ? |
|
Kind off, only to be used when remote service is unreachable as a fallback. Given that disks are not to be considered durable, a true (Always used) store and forward mechanism would almost guarantee message loss. |
|
What kind of unreachable remote service do you have in mind ?
|
|
Yes, but it's useful in many scenarios as in big networks, like azure, network partitioning is bound to happen once in a while. |
|
And the more mundane use case for which this is a prerequisite is the ability to deal with start/stop feature on SB queues/topics |
|
Since, as you mentioned, local hard drives are not durable, and the durable hard drives rely on Storage (which is the primary candidate for an unreachable remote service) - any solution would be undurable. Correct ? |
|
Yes, any solution would have a chance to loose messages, but the more we combine this kind of solutions the less likely it will become |
|
Agree there's no bulletproof solution. Tying together statistically reliable services (3 9's+) may be good enough, but we need to consider the scenarios.
Thoughts ? additional scenarios ? |
|
I don't think SB depends on storage, it depends on sql azure. And only IAAS VM's will be recycled & moved to the failover replica's (and loose a few minutes of data in the process), cloudservices vm's would not be recycled as their disk isn't persisted to begin with. (Exception ofcourse if the user's code is using storage, which is also very likely obviously) |
|
Other scenario's is when using other/multiple transports, f.e. rabbit hosted on a vm but that vm is unreachable |
|
So we have a matrix of scenarios, some which may benefit from local storage and some will not. Can you create a a matrix indicating that ? (this way, we can clearly evaluate the options / cost / value of this feature) |
|
We could use this feature, I am not sure how this will plug into the Outbox work being in NSB.Core. I would prefer to see some sort of MSMQ outbox. I was also thinking for performance reasons to actually use MSMQ as our sink and then run a background thread to shovel message from MSMQ to ASB using the native ASB batch api. We should be able to get much higher throughput albeit a little more latency, and we could also get offer an MS-DTC transaction with our on premise applications. |
|
I wrote an outbox persister that stores to azure in-role shared caching to tackle this. |
|
@jhashemi that doesn't sound like a great idea, there is absolutely no guarantee that a cache (regardless which one) will hold on to your data... |
SeanFeldman
changed the title
|
A PlatDev issue to replace this one. |
|
Closing this issue for now. We are tracking this requested functionality internally. |
4/22/2014 9:08:00 AM +00:00
Originally opened as Particular/NServiceBus.Azure#136
Provide a local outbox (in memory or on disk), that can be used to store messages while a remote service is unreachable
The text was updated successfully, but these errors were encountered: