Transaction causes a DisposedObjectException during a retry scenario

[adrianwright]

Is this a bug report?

Yes

Can you also reproduce the problem with the lastest version?

Yes

Environment

1. Operating system: Windows 10
2. Visual Studio version: 2017 Pro
3. Dotnet version: 4.6.2

Steps to Reproduce

1. Configure a bus for either in-memory or RabbitMQ transport (have not tried other transports)
2. Enable transactions and retries
3. Process a message that causes a retry
4. When the retry occurs, accessing the transaction results in a disposed object exception.

Expected Behavior

I expected a new transaction object to be available

Actual Behavior

A DisposedObjectException was thrown

Reproducible Demo

You will see in this demo that the errors occurs when we try to access the transaction during the retry.

class Program
    {
        static void Main(string[] args)
        {

            var bus = Bus.Factory.CreateUsingRabbitMq(config =>
            {

                var host = config.Host(new Uri("rabbitmq://my-rabbit"), hst =>

                {
                    hst.Username("guest");
                    hst.Password("guest");
                });
                
                config.UseRetry(x => x.Interval(5, new TimeSpan(0, 0, 30)));
                config.UseTransaction(x =>
                {
                    x.Timeout = TimeSpan.FromSeconds(90);

                    x.IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted;


                });
                config.ReceiveEndpoint(host, "testRetry", e =>
                {
                    e.Consumer<Consumer>();
                });


            });
            
            bus.Start();




            var sendToUri = new Uri("rabbitmq://my-rabbit/testRetry");
            var endPoint = bus.GetSendEndpoint(sendToUri).Result;
            endPoint.Send<Message>(new Message {Id = Guid.NewGuid()});
        }


        public class Consumer : IConsumer<Message>
        {
            public Task Consume(ConsumeContext<Message> context)
            {
                TransactionContext transactionContext;
                context.TryGetPayload(out transactionContext);


                // verify that I have a valid transaction. On the first retry, this line will throw an exception because the transaction has been disposed.
                var isolationLevel = transactionContext.Transaction.IsolationLevel;


                throw new Exception("throwing an exception to trigger a retry");
            }
        }


        public class Message
        {
            public Guid Id { get; set; }
        }
    }

[paveldayneko]

Facing the same issue.
Also, if you try to change order of registering middlewares and register UseTransaction before UseRetry you will receive exactly the same transaction on every retry. It won't be commited or rolled back. Just exactly the same.
As a workaround you can use message redelivering. It works fine with transactions

[Yavari]

Is this issue resolved? We are having the same problem.

[wasabii]

@phatboyg This seems to still happen.

Hypothesis: When backing out of the transaction filter scope, the transaction isn't removed from the payload. Then, when entering again, it already exists, and thus isn't created anew.

[phatboyg]

Order matters.

x.UseRetry(...);
x.UseTransaction(...);

This should work properly, and create a new transaction for each retry attempt.

[wasabii]

Yeah, that's what we're doing. It does not work for us.

Looking at the TransactionFilter code, I see that it doesn't seem to discard the payload when exiting the filter. I don't know much about payloads.... are they automatically discarded when the pipe exits up a level? If not, then AddOrCreate won't run a second time.

[phatboyg]

With Retry it will, because the retry creates a scope for each attempt, discarding anything that was added or updated within the attempt.

[phatboyg]

This is resolved, isn't it?

[wasabii]

It was not resolved for me at last check, no.

I will try to reproduce again tomorrow or the next day for you, however.

[phatboyg]

@wasabii if you still see an issue, please post the bus/endpoint configuration so that I can verify it before spending any time on it.

[phatboyg]

@wasabii any luck reproducing it with the latest /develop build?

[phatboyg]

Also, this was verified using the unit test based on the issue:
https://github.com/MassTransit/MassTransit/blob/develop/src/MassTransit.Tests/Pipeline/Transaction_Specs.cs#L151