[Transactionn] Transaction admin api getTransactionBufferStatus and getPendingAckStatus #10650

congbobo184 · 2021-05-20T06:52:48Z

Motivation

Transaction add admin api getTransactionBufferStatus
Transaction add admin api getPendingAckStatus

implement

    @Data
    public static class TransactionBufferStatus {
        /** The state of this transaction buffer. */
        public String state;

        /** The max read position of this transaction buffer. */
        public String maxReadPosition;

        /** The last snapshot timestamps of this transaction buffer. */
        public long lastSnapshotTimestamps;
    }

    @Data
    public static class TransactionPendingAckStatus{
        /** The state of this pending ack. */
        public String state;
    }

This is transaction component status.

Verifying this change

Add the tests for it

Does this pull request potentially affect one of the following parts:
If yes was chosen, please highlight the changes

Dependencies (does it add or upgrade a dependency): (no)
The public API: (no)
The schema: (no)
The default values of configurations: (no)
The wire protocol: (no)
The rest endpoints: (no)
The admin cli options: (yes)
Anything that affects deployment: (no)

…_admin_api # Conflicts: # pulsar-client-tools-test/src/test/java/org/apache/pulsar/admin/cli/PulsarAdminToolTest.java

…_admin_api

…nto congbobo184_transaction_admin_api_component_in_topic_status

…t_in_topic_status # Conflicts: # pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/TransactionsBase.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v3/Transactions.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentSubscription.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/TransactionBuffer.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/InMemTransactionBuffer.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TopicTransactionBuffer.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/buffer/impl/TransactionBufferDisable.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/pendingack/PendingAckHandle.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/pendingack/impl/PendingAckHandleDisabled.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/transaction/pendingack/impl/PendingAckHandleImpl.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v3/AdminApiTransactionTest.java # pulsar-client-admin-api/src/main/java/org/apache/pulsar/client/admin/Transactions.java # pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/TransactionsImpl.java # pulsar-client-tools-test/src/test/java/org/apache/pulsar/admin/cli/PulsarAdminToolTest.java # pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTransactions.java

eolivelli · 2021-05-25T11:49:03Z

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v3/Transactions.java

+            @ApiResponse(code = 404, message = "Tenant or cluster or namespace or topic doesn't exist"),
+            @ApiResponse(code = 503, message = "This Broker is not configured "
+                    + "with transactionCoordinatorEnabled=true."),
+            @ApiResponse(code = 307, message = "Topic don't owner by this broker!"),


what about
"Topic is not owned by this broker"

eolivelli · 2021-05-25T11:49:44Z

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v3/Transactions.java

+            @ApiResponse(code = 503, message = "This Broker is not configured "
+                    + "with transactionCoordinatorEnabled=true."),
+            @ApiResponse(code = 307, message = "Topic don't owner by this broker!"),
+            @ApiResponse(code = 501, message = "Topic is not a persistent topic!"),


"Topic is not a persistent topic"

501 is kind of "internal server error", probably we should return something like "Not allowed" or "Bad request"

eolivelli · 2021-05-25T11:51:11Z

pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v3/Transactions.java

+    public void getTransactionInBufferStats(@Suspended final AsyncResponse asyncResponse,
+                                            @QueryParam("authoritative")
+                                            @DefaultValue("false") boolean authoritative,
+                                            @QueryParam("mostSigBits")


isn't it a little bit awkward to to pass separately mostSigBits and leastSigBits in a REST API ?
can we using a string representation ?

do we have other examples in the REST API about splitting the TxID into the two parts ?

client don't know the txnId is consists of coordinatorId and sequenceId. it only know a TxnID consists of mostSigBits and leastSigBits. It's better to understand.

@eolivelli Currently all the txn Id in the admin API expressed as mostSigBits and leastSigBits. The client also can get the mostSigBits and leastSigBits of a transactionId when using the transactions.

codelipenghui

Please check all the name which use **Status, we are using **Stats before.

codelipenghui · 2021-05-25T23:56:48Z

pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/TransactionBufferStatus.java

+ */
+package org.apache.pulsar.common.policies.data;
+
+public class TransactionBufferStatus {


Suggested change

public class TransactionBufferStatus {

public class TransactionBufferStats {

codelipenghui · 2021-05-25T23:57:10Z

...common/src/main/java/org/apache/pulsar/common/policies/data/TransactionPendingAckStatus.java

+ */
+package org.apache.pulsar.common.policies.data;
+
+public class TransactionPendingAckStatus {


Suggested change

public class TransactionPendingAckStatus {

public class TransactionPendingAckStats {

codelipenghui · 2021-05-25T23:57:41Z

pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTransactions.java

@@ -42,6 +42,31 @@ void run() throws Exception {
        }
    }

+    @Parameters(commandDescription = "Get transaction buffer status")
+    private class GetTransactionBufferStatus extends CliCommand {


Suggested change

private class GetTransactionBufferStatus extends CliCommand {

private class GetTransactionBufferStats extends CliCommand {

codelipenghui · 2021-05-25T23:57:50Z

pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTransactions.java

+    }
+
+    @Parameters(commandDescription = "Get transaction pending ack status")
+    private class GetPendingAckStatus extends CliCommand {


Suggested change

private class GetPendingAckStatus extends CliCommand {

private class GetPendingAckStats extends CliCommand {

codelipenghui · 2021-05-25T23:58:07Z

pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTransactions.java

@@ -84,6 +109,8 @@ void run() throws Exception {
    public CmdTransactions(Supplier<PulsarAdmin> admin) {
        super("transactions", admin);
        jcommander.addCommand("coordinator-status", new GetCoordinatorStatus());
+        jcommander.addCommand("transaction-buffer-status", new GetTransactionBufferStatus());


Suggested change

jcommander.addCommand("transaction-buffer-status", new GetTransactionBufferStatus());

jcommander.addCommand("transaction-buffer-stats", new GetTransactionBufferStats());

codelipenghui · 2021-05-25T23:58:16Z

pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTransactions.java

@@ -84,6 +109,8 @@ void run() throws Exception {
    public CmdTransactions(Supplier<PulsarAdmin> admin) {
        super("transactions", admin);
        jcommander.addCommand("coordinator-status", new GetCoordinatorStatus());
+        jcommander.addCommand("transaction-buffer-status", new GetTransactionBufferStatus());
+        jcommander.addCommand("pending-ack-status", new GetPendingAckStatus());


Suggested change

jcommander.addCommand("pending-ack-status", new GetPendingAckStatus());

jcommander.addCommand("pending-ack-stats", new GetPendingAckStats());

…t_in_topic_status # Conflicts: # pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/TransactionsBase.java # pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v3/Transactions.java # pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/v3/AdminApiTransactionTest.java # pulsar-client-admin-api/src/main/java/org/apache/pulsar/client/admin/Transactions.java # pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/TransactionsImpl.java # pulsar-client-tools-test/src/test/java/org/apache/pulsar/admin/cli/PulsarAdminToolTest.java

congbobo184 · 2021-05-26T02:40:52Z

@Anonymitaet please review doc thanks.

Anonymitaet · 2021-05-26T03:12:22Z

pulsar-client-admin-api/src/main/java/org/apache/pulsar/client/admin/Transactions.java

+     * Get transaction pending ack stats.
+     *
+     * @param topic the topic of this transaction pending ack stats
+     * @param subName the topic of this transaction pending ack stats


Suggested change

* @param subName the topic of this transaction pending ack stats

* @param subName the subName of this transaction pending ack stats

is subName accurate? or subscription name?

codelipenghui · 2021-05-26T07:56:01Z

@congbobo184 Could you please update the title of the PR?

eolivelli

perfect

…etPendingAckStatus (apache#10650)

congbo added 7 commits May 19, 2021 16:54

[Transaction] Transaction admin api get transaction coordinator status.

6024aee

Merge branch 'master' into congbobo184_transaction_coordinator_status…

e79611f

…_admin_api # Conflicts: # pulsar-client-tools-test/src/test/java/org/apache/pulsar/admin/cli/PulsarAdminToolTest.java

Fix some test

ef4dde2

Fix some comment

3226755

[Transaction] Transaction admin api component in topic status.

87f4b1a

Merge branch 'master' into congbobo184_transaction_coordinator_status…

5f03e8b

…_admin_api

Merge branch 'congbobo184_transaction_coordinator_status_admin_api' i…

7c89472

…nto congbobo184_transaction_admin_api_component_in_topic_status

Anonymitaet added the doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. label May 20, 2021

congbobo184 requested review from eolivelli and codelipenghui May 20, 2021 09:49

congbobo184 self-assigned this May 20, 2021

congbobo184 added this to the 2.8.0 milestone May 21, 2021

congbo added 3 commits May 22, 2021 01:10

Fix some comment

730ebe1

Fix exception to 307

8afd657

eolivelli requested changes May 25, 2021

View reviewed changes

codelipenghui requested changes May 26, 2021

View reviewed changes

Fix some code style

ae29aac

Anonymitaet reviewed May 26, 2021

View reviewed changes

Fix some code style

399c1bb

codelipenghui approved these changes May 26, 2021

View reviewed changes

congbobo184 changed the title ~~[Transactionn] Transaction admin api component in topic status~~ [Transactionn] Transaction admin api getTransactionBufferStatus and getPendingAckStatus May 26, 2021

eolivelli approved these changes May 26, 2021

View reviewed changes

eolivelli merged commit 3b504d0 into apache:master May 26, 2021

yangl pushed a commit to yangl/pulsar that referenced this pull request Jun 23, 2021

[Transactionn] Transaction admin api getTransactionBufferStatus and g…

8ea88e9

…etPendingAckStatus (apache#10650)

bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022

[Transactionn] Transaction admin api getTransactionBufferStatus and g…

53eccb1

…etPendingAckStatus (apache#10650)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Transactionn] Transaction admin api getTransactionBufferStatus and getPendingAckStatus #10650

[Transactionn] Transaction admin api getTransactionBufferStatus and getPendingAckStatus #10650

congbobo184 commented May 20, 2021 •

edited

Loading

eolivelli May 25, 2021

eolivelli May 25, 2021

eolivelli May 25, 2021

congbobo184 May 25, 2021

codelipenghui May 26, 2021

codelipenghui left a comment

codelipenghui May 25, 2021

codelipenghui May 25, 2021

codelipenghui May 25, 2021

codelipenghui May 25, 2021

codelipenghui May 25, 2021

codelipenghui May 25, 2021

congbobo184 commented May 26, 2021 •

edited

Loading

Anonymitaet May 26, 2021

codelipenghui commented May 26, 2021

eolivelli left a comment

	public class TransactionBufferStatus {
	public class TransactionBufferStats {

	public class TransactionPendingAckStatus {
	public class TransactionPendingAckStats {

	private class GetTransactionBufferStatus extends CliCommand {
	private class GetTransactionBufferStats extends CliCommand {

	private class GetPendingAckStatus extends CliCommand {
	private class GetPendingAckStats extends CliCommand {

	jcommander.addCommand("transaction-buffer-status", new GetTransactionBufferStatus());
	jcommander.addCommand("transaction-buffer-stats", new GetTransactionBufferStats());

	jcommander.addCommand("pending-ack-status", new GetPendingAckStatus());
	jcommander.addCommand("pending-ack-stats", new GetPendingAckStats());

	* @param subName the topic of this transaction pending ack stats
	* @param subName the subName of this transaction pending ack stats

[Transactionn] Transaction admin api getTransactionBufferStatus and getPendingAckStatus #10650

[Transactionn] Transaction admin api getTransactionBufferStatus and getPendingAckStatus #10650

Conversation

congbobo184 commented May 20, 2021 • edited Loading

Motivation

implement

Verifying this change

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codelipenghui left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

congbobo184 commented May 26, 2021 • edited Loading

Choose a reason for hiding this comment

codelipenghui commented May 26, 2021

eolivelli left a comment

Choose a reason for hiding this comment

congbobo184 commented May 20, 2021 •

edited

Loading

congbobo184 commented May 26, 2021 •

edited

Loading