feat: implement mria:sync_transaction/4,3,2 #136

SergeTupchiy · 2023-04-06T18:32:15Z

When called on a replicant node, this function blocks a caller process until the transaction is replicated on that node or a timeout occurs. Otherwise, it behaves the same way as mria:transaction/3,2

Closes: EMQX-9102

src/mria_upstream.erl

src/mria.erl

When called on a replicant node, this function blocks a caller process until the transaction is replicated on that node or a timeout occurs. Otherwise, it behaves the same way as mria:transaction/3,2 Closes: EMQX-9102

ieQu1 · 2023-04-13T17:56:47Z

src/mria.erl

@@ -600,3 +639,69 @@ db_nodes_maybe_rpc() ->
                    []
            end
    end.
+
+sync_replicant_trans(Shard, Function, Args, ReplTimeout) ->
+    case mria_rlog:wait_for_shards([Shard], ReplTimeout) of


I think we should actually measure time spent during this stage, as well as other stages, and do something

TimeLeft = ReplTimeout - TWaitingForShards, ... maybe_retry_waiting(..., TimeLeft)

in order to not exceed the user-specified timeout value.

Also thought about it, will definitely add it.

Done.
Please note that I deliberately haven't measured rpc_to_core_node time, since it uses retries / cool down based on mria_config values.
If we want to limit total function execution to a user-specified timeout, I'll also update rpc_to_core_node implementation.

src/mria.erl

src/mria_rlog.hrl

ieQu1 · 2023-04-13T18:24:41Z

src/mria_upstream.erl

+    mria_rlog:wait_for_shards([Shard], infinity),
+    mnesia:transaction(fun() ->
+                               Res = apply(Fun, Args),
+                               ok = mnesia:write(ReplyTo),


I wonder if it's possible to inject this write on the replicant side? It will lead to better code reuse, since it will use the existing transactional_wrapper.

Yes, thanks for a good suggestion, fixed:

SyncFun = fun(Args_) -> Res_ = apply(Fun, Args_), ok = mnesia:write(ReplyTo), Res_ end, CoreRes = rpc_to_core_node(Shard, mria_upstream, transactional_wrapper, [Shard, SyncFun, [Args]]),

ieQu1 · 2023-04-13T20:07:14Z

src/mria.erl

@@ -368,6 +371,42 @@ ro_transaction(Shard, Fun) ->
            end
    end.

+%% @doc Synchronous transaction.
+%% This function has a behavior different from transaction/2,3 only when called on a replicant node.


Suggested change

%% This function has a behavior different from transaction/2,3 only when called on a replicant node.

%% This function has a behavior different from `transaction/2,3' only when called on a replicant node.

ieQu1 · 2023-04-13T20:07:28Z

src/mria.erl

@@ -368,6 +371,42 @@ ro_transaction(Shard, Fun) ->
            end
    end.

+%% @doc Synchronous transaction.
+%% This function has a behavior different from transaction/2,3 only when called on a replicant node.
+%% When a process on a replicant node calls sync_transaction/4,3,2, it will be blocked


Suggested change

%% When a process on a replicant node calls sync_transaction/4,3,2, it will be blocked

%% When a process on a replicant node calls `sync_transaction/4,3,2', it will be blocked

ieQu1 · 2023-04-13T20:07:45Z

src/mria.erl

+%% When a process on a replicant node calls sync_transaction/4,3,2, it will be blocked
+%% until the transaction is replicated on that node or a timeout occurs.
+%% It should be noted that a ReplTimeout doesn't control the total maximum execution time
+%% of sync_transaction/4,3,2. It is only used to wait for a transaction to be imported


Suggested change

%% of sync_transaction/4,3,2. It is only used to wait for a transaction to be imported

%% of `sync_transaction/4,3,2'. It is only used to wait for a transaction to be imported

ieQu1 · 2023-04-13T20:08:06Z

src/mria.erl

+%% to the local node which originated the transaction.
+%% Thus, the total execution time may be significantly higher,
+%% e. g. when rpc to core node was slow and/or retried.
+%% Moreover, when {timeout, t_result(A)} is returned, the result can be successful.


Suggested change

%% Moreover, when {timeout, t_result(A)} is returned, the result can be successful.

%% Moreover, when `{timeout, t_result(A)}' is returned, the result can be successful.

src/mria.erl

src/mria_schema.erl

ieQu1 · 2023-04-21T16:09:03Z

src/mria.erl

@@ -368,6 +371,42 @@ ro_transaction(Shard, Fun) ->
            end
    end.

+%% @doc Synchronous transaction.
+%% This function has a behavior different from 'transaction/2,3' only when called on a replicant node.


Small remark for the future: edoc uses LaTeX-style quoting for code: `' rather than ''.
Let's ignore it, since we don't build edoc for mria

SergeTupchiy self-assigned this Apr 6, 2023

SergeTupchiy commented Apr 6, 2023

View reviewed changes

src/mria_upstream.erl Show resolved Hide resolved

SergeTupchiy commented Apr 6, 2023

View reviewed changes

src/mria.erl Show resolved Hide resolved

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch 6 times, most recently from 9070b76 to e03e83e Compare April 11, 2023 19:38

SergeTupchiy commented Apr 11, 2023

View reviewed changes

src/mria.erl Outdated Show resolved Hide resolved

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch 2 times, most recently from a07b662 to 739bcb8 Compare April 12, 2023 11:40

SergeTupchiy marked this pull request as ready for review April 12, 2023 11:42

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch from 739bcb8 to 88ce38d Compare April 12, 2023 13:05

SergeTupchiy marked this pull request as draft April 12, 2023 16:13

feat: implement mria:sync_transaction/4,3,2

02c269b

When called on a replicant node, this function blocks a caller process until the transaction is replicated on that node or a timeout occurs. Otherwise, it behaves the same way as mria:transaction/3,2 Closes: EMQX-9102

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch from 88ce38d to 02c269b Compare April 12, 2023 20:19

SergeTupchiy marked this pull request as ready for review April 12, 2023 20:23

SergeTupchiy removed their assignment Apr 12, 2023

ieQu1 reviewed Apr 13, 2023

View reviewed changes

src/mria.erl Show resolved Hide resolved

ieQu1 reviewed Apr 13, 2023

View reviewed changes

src/mria_rlog.hrl Outdated Show resolved Hide resolved

ieQu1 reviewed Apr 13, 2023

View reviewed changes

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch from 65f37f7 to 53f9a2c Compare April 13, 2023 19:05

ieQu1 reviewed Apr 13, 2023

View reviewed changes

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch 2 times, most recently from bca4d30 to ab50fe6 Compare April 14, 2023 18:17

thalesmg reviewed Apr 17, 2023

View reviewed changes

src/mria.erl Show resolved Hide resolved

src/mria_schema.erl Outdated Show resolved Hide resolved

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch from ab50fe6 to 2e7fe50 Compare April 18, 2023 07:56

chore(mria_membership): fix indention

9dcfae2

SergeTupchiy force-pushed the EMQX-9102-sync-transaction-replicant branch from 2e7fe50 to 9dcfae2 Compare April 18, 2023 20:02

ieQu1 reviewed Apr 21, 2023

View reviewed changes

ieQu1 approved these changes Apr 21, 2023

View reviewed changes

SergeTupchiy merged commit 01b66cc into emqx:main Apr 24, 2023

p-linnane mentioned this pull request Apr 26, 2023

emqx 5.0.24 Homebrew/homebrew-core#129435

Merged

ieQu1 mentioned this pull request Jun 21, 2023

Monotonic reads violation #2

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement mria:sync_transaction/4,3,2 #136

feat: implement mria:sync_transaction/4,3,2 #136

SergeTupchiy commented Apr 6, 2023

ieQu1 Apr 13, 2023

SergeTupchiy Apr 13, 2023

SergeTupchiy Apr 14, 2023 •

edited

Loading

ieQu1 Apr 13, 2023 •

edited

Loading

SergeTupchiy Apr 14, 2023

ieQu1 Apr 13, 2023

SergeTupchiy Apr 14, 2023

ieQu1 Apr 13, 2023

SergeTupchiy Apr 14, 2023

ieQu1 Apr 13, 2023

SergeTupchiy Apr 14, 2023

ieQu1 Apr 13, 2023

SergeTupchiy Apr 14, 2023

ieQu1 Apr 21, 2023

	%% This function has a behavior different from transaction/2,3 only when called on a replicant node.
	%% This function has a behavior different from `transaction/2,3' only when called on a replicant node.

	%% When a process on a replicant node calls sync_transaction/4,3,2, it will be blocked
	%% When a process on a replicant node calls `sync_transaction/4,3,2', it will be blocked

	%% of sync_transaction/4,3,2. It is only used to wait for a transaction to be imported
	%% of `sync_transaction/4,3,2'. It is only used to wait for a transaction to be imported

	%% Moreover, when {timeout, t_result(A)} is returned, the result can be successful.
	%% Moreover, when `{timeout, t_result(A)}' is returned, the result can be successful.

feat: implement mria:sync_transaction/4,3,2 #136

feat: implement mria:sync_transaction/4,3,2 #136

Conversation

SergeTupchiy commented Apr 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SergeTupchiy Apr 14, 2023 • edited Loading

Choose a reason for hiding this comment

ieQu1 Apr 13, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SergeTupchiy Apr 14, 2023 •

edited

Loading

ieQu1 Apr 13, 2023 •

edited

Loading