[Bug] [txn] callback of commit/abort is not accurate in transaction

[TakaHiR07]

Search before asking

• I searched in the issues and found nothing similar.

Version

master branch

Minimal reproduce step

When I reading the transaction code and pip-31，I found the implementation of CommandEndTxn is not as pip-31 described. Now the txn client execute commit() or abort() would wait the reponse from server until txn status become committed or aborted.

And now the callback of commit() can not reflect the actual status of txn. For example, 2 case may occur

1. broker may be unavailable before server update txnstatus to committing
2. broker may be unavailable between committing and committed

For case 2, may throw CoordinatorNotFound exception to client, then client would retry txn.commit(). But when broker recover, coordinator would firstly recover the previous transactions. Then broker process the retry-commit, may return InvalidTxnStatus or transactionNotFound to client, Which client would regard txn commit error, but actually it is success.

Why we don't respond success to client when server update txn-status to committing ? @congbobo184 @liangyepianzhou

What did you expect to see?

throw exception in commit.callback, but txn is success

What did you see instead?

commit.callback can reflect the accurate status

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[congbobo184]

@TakaHiR07 hi, let's see this step

1. client sends an end request to the broker
2. broker changes the status to committing or aborting status success
3. broker down, can't send any response to the client
4. client also doesn't know the end request success or fail(when tc reload will reprocess this txn, the client will receive TransationNorFoundException but we don't know it ended by timeout or by client end request)

I think your issue seems like #19629

of course, we can improve it, when txn status changes to committing or aborting status success then send the response to the client.

[TakaHiR07]

@congbobo184 I agree that we can improve it. When txn status successfully become committing or aborting, client can regard the transaction commit/abort successfully. Then the process of committing -> committed or aborting -> aborted is an Inevitable process, which is ensured by pulsar-server, just as kafka implemented and the pip-31 described.
And I will try to push a pr to improve it soon.

[github-actions]

The issue had no activity for 30 days, mark with Stale label.