Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Topic level policy support in pulsarctl #246

Open
24 of 61 tasks
codelipenghui opened this issue Aug 12, 2020 · 22 comments
Open
24 of 61 tasks

[Feature] Topic level policy support in pulsarctl #246

codelipenghui opened this issue Aug 12, 2020 · 22 comments

Comments

@codelipenghui
Copy link
Contributor

codelipenghui commented Aug 12, 2020

The topic level policy is planned to release 2.7.0 and there are some PRs already merged into the Pulsar master branch. The process of the topic level policy support in Pulsar can be found at apache/pulsar#2688. It's better to support topic level policy in pulsarctl.

Append comand

  • 15. clear-backlog
  • 16. compaction-status
  • 17. create-missed-partitions
  • 18. create-partitioned-topic
  • 19. delete-partitioned-topic
  • 20. examine-messages
  • 21. expire-messages-all-subscriptions
  • 22. get-deduplication-snapshot-interval
  • 23. get-max-consumers-per-subscription
  • 24. get-max-message-size
  • 25. get-max-subscriptions
  • 26. get-message-by-id
  • 27. get-message-id
  • 28. get-partitioned-topic-metadata
  • 29. get-replicator-dispatch-rate
  • 30. get-subscribe-rate
  • 31. get-subscription-dispatch-rate
  • 32. get-subscription-types-enabled
  • 33. grant-permission
  • 34. info-internal
  • 35. list-partitioned-topics
  • 36. partitioned-lookup
  • 37. partitioned-stats
  • 38. partitioned-stats-internal
  • 39. permissions
  • 40. remove-deduplication-snapshot-interval
  • 41. remove-max-consumers-per-subscription
  • 42. remove-max-message-size
  • 43. remove-max-subscriptions
  • 44. remove-replicator-dispatch-rate
  • 45. remove-subscribe-rate
  • 46. remove-subscription-dispatch-rate
  • 47. revoke-permission
  • 48. set-deduplication-snapshot-interval
  • 49. set-max-consumers-per-subscription
  • 50. set-max-message-size
  • 51. set-max-subscriptions
  • 52. set-replicated-subscription-status
  • 53. set-replicator-dispatch-rate
  • 54. set-subscribe-rate
  • 55. set-subscription-dispatch-rate
  • 56. set-subscription-types-enabled
  • 57. stats-internal
  • 58. truncate
  • 59. update-partitioned-topic

Waiting for confirmation:

@codelipenghui codelipenghui changed the title Topic level policy support in pulsarctl [Feature] Topic level policy support in pulsarctl Aug 12, 2020
@limingnihao
Copy link
Contributor

limingnihao commented May 27, 2021

I will add MessageTTL [Get, Set, Remove].

limingnihao pushed a commit to limingnihao/pulsarctl that referenced this issue May 28, 2021
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue May 28, 2021
@limingnihao
Copy link
Contributor

I will add Max Producer [Get, Set, Remove], and Max Consumer [Get, Set, Remove].

codelipenghui pushed a commit that referenced this issue Jun 2, 2021
Add command topic message ttl:
* pulsarctl topics get-message-ttl [topic]
* pulsarctl topics set-message-ttl [topic] -t [seconds] 
* pulsarctl topics remove-message-ttl [topic]
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 2, 2021
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 2, 2021
codelipenghui pushed a commit that referenced this issue Jun 2, 2021
Add command topic max number of producers:

* pulsarctl topics get-max-producers [topic]
* pulsarctl topics set-max-producers [topic] -p [max]
* pulsarctl topics remove-max-producers [topic]
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 2, 2021
Add command topic max number of producers:

* pulsarctl topics get-max-producers [topic]
* pulsarctl topics set-max-producers [topic] -p [max]
* pulsarctl topics remove-max-producers [topic]
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 2, 2021
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 2, 2021
limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 2, 2021
zymap pushed a commit that referenced this issue Jun 2, 2021
Add command topic max number of consumers:

* pulsarctl topics get-max-consumers [topic]
* pulsarctl topics set-max-consumers [topic] -c [max]
* pulsarctl topics remove-max-consumers [topic]
@limingnihao
Copy link
Contributor

I will add MaxUnackMessagesPerConsumer [Get, Set, Remove].

limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 3, 2021
zymap pushed a commit that referenced this issue Jun 3, 2021
Add command topic max unacked messages per consumer:

* pulsarctl topics get-max-unacked-messages-per-consumer [topic]
* pulsarctl topics set-max-unacked-messages-per-consumer [topic] -m [max]
* pulsarctl topics remove-max-unacked-messages-per-consumer [topic]
@limingnihao
Copy link
Contributor

I will add MaxUnackMessagesPerSubscription [Get, Set, Remove]

limingnihao pushed a commit to limingnihao/pulsarctl that referenced this issue Jun 7, 2021
codelipenghui pushed a commit that referenced this issue Jun 7, 2021
Add command topic max unacked messages per subscription:

* pulsarctl topics get-max-unacked-messages-per-subscription [topic]
* pulsarctl topics set-max-unacked-messages-per-subscription [topic] -m [max]
* pulsarctl topics remove-max-unacked-messages-per-subscription [topic]
@limingnihao
Copy link
Contributor

I will add Persistence Policy [Get, Set, Remove]

limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 8, 2021
@limingnihao
Copy link
Contributor

I will add Delayed Delivery Policies [Get, Set, Remove]

limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 16, 2021
zymap pushed a commit that referenced this issue Jun 18, 2021
Add command topic Delayed Delivery Policies:

- pulsarctl topics get-delayed-delivery [topic]
- pulsarctl topics set-delayed-delivery [topic] -t 22s -e
- pulsarctl topics remove-delayed-delivery [topic]
@limingnihao
Copy link
Contributor

limingnihao commented Jun 21, 2021

I will add Backlog Quota [Get, Set, Remove]

limingnihao added a commit to limingnihao/pulsarctl that referenced this issue Jun 24, 2021
Signed-off-by: limingnihao <limingnihao@live.com>
nodece added a commit that referenced this issue Nov 12, 2021
background from #246,  the PR implements the following commands:

- `pulsar topics get-retention <topic> -a` - Get the retention policy for a topic
- `pulsar topics remove-retention <topic>` - Remove the retention policy for a topic
- `pulsar topics set-retention <topic> --time <string> --size <string>` - Set the retention policy for a topic

(cherry picked from commit c4edd49)
nodece added a commit that referenced this issue Nov 12, 2021
### Changes

background from #246,  the PR implements the following commands:

- `pulsarctl topics get-compaction-threshold <topic> --applied <bool>` - Get the compaction threshold for a topic
- `pulsarctl topics remove-compaction-threshold <topic>` - Remove the compaction threshold for a topic
- `pulsarctl topics set-compaction-threshold <topic> --threshold <string>` Set the compaction threshold for a topic

(cherry picked from commit a8286d1)
nodece added a commit that referenced this issue Nov 12, 2021
Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Changes

background from #246,  the PR implements the following commands:

- `pulsarctl topics get-backlog-quotas <topic> -a` - Get the backlog quota policy for a topic
- `pulsarctl topics remove-backlog-quota <topic> --type <producer_request_hold|message_age>` - Remove a backlog quota policy from a topic
- `pulsarctl topics set-backlog-quota <topic> --limit-size <string> --limit-time <int> --policy <producer_request_hold|producer_exception|consumer_backlog_eviction> --type <producer_request_hold|message_age>` - Set a backlog quota policy for a topic

### TODO
- [x] Add integration tests

(cherry picked from commit c58eef0)
nodece added a commit that referenced this issue Nov 12, 2021
Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Changes

background from #246,  the PR implements the following commands:

- `pulsarctl topics get-inactive-topic-policies <topic> --applied <bool>`
- `pulsarctl topics remove-inactive-topic-policies <topic>`
- `pulsarctl topics set-inactive-topic-policies <topic> --enable-delete-while-inactive <bool> --max-inactive-duration <string> --delete-mode <delete_when_no_subscriptions|delete_when_subscriptions_caught_up>`

(cherry picked from commit db4dda4)
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
Add command topic message ttl:
* pulsarctl topics get-message-ttl [topic]
* pulsarctl topics set-message-ttl [topic] -t [seconds] 
* pulsarctl topics remove-message-ttl [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…#353)

Add command topic max number of producers:

* pulsarctl topics get-max-producers [topic]
* pulsarctl topics set-max-producers [topic] -p [max]
* pulsarctl topics remove-max-producers [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…#355)

Add command topic max number of consumers:

* pulsarctl topics get-max-consumers [topic]
* pulsarctl topics set-max-consumers [topic] -c [max]
* pulsarctl topics remove-max-consumers [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…lsarctl#246) (apache#358)

Add command topic max unacked messages per consumer:

* pulsarctl topics get-max-unacked-messages-per-consumer [topic]
* pulsarctl topics set-max-unacked-messages-per-consumer [topic] -m [max]
* pulsarctl topics remove-max-unacked-messages-per-consumer [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…e/pulsarctl#246) (apache#361)

Add command topic max unacked messages per subscription:

* pulsarctl topics get-max-unacked-messages-per-subscription [topic]
* pulsarctl topics set-max-unacked-messages-per-subscription [topic] -m [max]
* pulsarctl topics remove-max-unacked-messages-per-subscription [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
) (apache#374)

Add command topic Delayed Delivery Policies:

- pulsarctl topics get-delayed-delivery [topic]
- pulsarctl topics set-delayed-delivery [topic] -t 22s -e
- pulsarctl topics remove-delayed-delivery [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…apache#397)

Add command topic Message Dispatch Rate:
> Pulsarctl does not support 2-letter shorthand, so use the full length.

- pulsarctl topics get-dispatch-rate [topic]
- pulsarctl topics set-dispatch-rate [topic] --msg-dispatch-rate 1 --byte-dispatch-rate 2 --dispatch-rate-period 3 --relative-to-publish-rate 
- pulsarctl topics remove-dispatch-rate [topic]
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
* Add command topic Deduplication(streamnative/pulsarctl#246)

- pulsarctl topics get-deduplication [topic]
- pulsarctl topics set-deduplication [topic] -e
- pulsarctl topics remove-deduplication [topic]

* Modify prompt

Signed-off-by: limingnihao <limingnihao@live.com>
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…tive/pulsarctl#420)

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsar topics get-retention <topic> -a` - Get the retention policy for a topic
- `pulsar topics remove-retention <topic>` - Remove the retention policy for a topic
- `pulsar topics set-retention <topic> --time <string> --size <string>` - Set the retention policy for a topic
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…ctl#430)

### Changes

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsarctl topics get-compaction-threshold <topic> --applied <bool>` - Get the compaction threshold for a topic
- `pulsarctl topics remove-compaction-threshold <topic>` - Remove the compaction threshold for a topic
- `pulsarctl topics set-compaction-threshold <topic> --threshold <string>` Set the compaction threshold for a topic
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Changes

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsarctl topics get-backlog-quotas <topic> -a` - Get the backlog quota policy for a topic
- `pulsarctl topics remove-backlog-quota <topic> --type <producer_request_hold|message_age>` - Remove a backlog quota policy from a topic
- `pulsarctl topics set-backlog-quota <topic> --limit-size <string> --limit-time <int> --policy <producer_request_hold|producer_exception|consumer_backlog_eviction> --type <producer_request_hold|message_age>` - Set a backlog quota policy for a topic

### TODO
- [x] Add integration tests
tisonkun pushed a commit to tisonkun/pulsar-client-go that referenced this issue Aug 15, 2023
…sarctl#444)

Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Changes

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsarctl topics get-inactive-topic-policies <topic> --applied <bool>`
- `pulsarctl topics remove-inactive-topic-policies <topic>`
- `pulsarctl topics set-inactive-topic-policies <topic> --enable-delete-while-inactive <bool> --max-inactive-duration <string> --delete-mode <delete_when_no_subscriptions|delete_when_subscriptions_caught_up>`
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
Add command topic message ttl:
* pulsarctl topics get-message-ttl [topic]
* pulsarctl topics set-message-ttl [topic] -t [seconds] 
* pulsarctl topics remove-message-ttl [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
Add command topic max number of producers:

* pulsarctl topics get-max-producers [topic]
* pulsarctl topics set-max-producers [topic] -p [max]
* pulsarctl topics remove-max-producers [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
Add command topic max number of consumers:

* pulsarctl topics get-max-consumers [topic]
* pulsarctl topics set-max-consumers [topic] -c [max]
* pulsarctl topics remove-max-consumers [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
…lsarctl#246) (#358)

Add command topic max unacked messages per consumer:

* pulsarctl topics get-max-unacked-messages-per-consumer [topic]
* pulsarctl topics set-max-unacked-messages-per-consumer [topic] -m [max]
* pulsarctl topics remove-max-unacked-messages-per-consumer [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
…e/pulsarctl#246) (#361)

Add command topic max unacked messages per subscription:

* pulsarctl topics get-max-unacked-messages-per-subscription [topic]
* pulsarctl topics set-max-unacked-messages-per-subscription [topic] -m [max]
* pulsarctl topics remove-max-unacked-messages-per-subscription [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
) (#374)

Add command topic Delayed Delivery Policies:

- pulsarctl topics get-delayed-delivery [topic]
- pulsarctl topics set-delayed-delivery [topic] -t 22s -e
- pulsarctl topics remove-delayed-delivery [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
…#397)

Add command topic Message Dispatch Rate:
> Pulsarctl does not support 2-letter shorthand, so use the full length.

- pulsarctl topics get-dispatch-rate [topic]
- pulsarctl topics set-dispatch-rate [topic] --msg-dispatch-rate 1 --byte-dispatch-rate 2 --dispatch-rate-period 3 --relative-to-publish-rate 
- pulsarctl topics remove-dispatch-rate [topic]
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
* Add command topic Deduplication(streamnative/pulsarctl#246)

- pulsarctl topics get-deduplication [topic]
- pulsarctl topics set-deduplication [topic] -e
- pulsarctl topics remove-deduplication [topic]

* Modify prompt

Signed-off-by: limingnihao <limingnihao@live.com>
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
…tive/pulsarctl#420)

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsar topics get-retention <topic> -a` - Get the retention policy for a topic
- `pulsar topics remove-retention <topic>` - Remove the retention policy for a topic
- `pulsar topics set-retention <topic> --time <string> --size <string>` - Set the retention policy for a topic
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
…ctl#430)

### Changes

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsarctl topics get-compaction-threshold <topic> --applied <bool>` - Get the compaction threshold for a topic
- `pulsarctl topics remove-compaction-threshold <topic>` - Remove the compaction threshold for a topic
- `pulsarctl topics set-compaction-threshold <topic> --threshold <string>` Set the compaction threshold for a topic
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Changes

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsarctl topics get-backlog-quotas <topic> -a` - Get the backlog quota policy for a topic
- `pulsarctl topics remove-backlog-quota <topic> --type <producer_request_hold|message_age>` - Remove a backlog quota policy from a topic
- `pulsarctl topics set-backlog-quota <topic> --limit-size <string> --limit-time <int> --policy <producer_request_hold|producer_exception|consumer_backlog_eviction> --type <producer_request_hold|message_age>` - Set a backlog quota policy for a topic

### TODO
- [x] Add integration tests
tisonkun pushed a commit to apache/pulsar-client-go that referenced this issue Aug 16, 2023
…sarctl#444)

Signed-off-by: Zixuan Liu <nodeces@gmail.com>

### Changes

background from streamnative/pulsarctl#246,  the PR implements the following commands:

- `pulsarctl topics get-inactive-topic-policies <topic> --applied <bool>`
- `pulsarctl topics remove-inactive-topic-policies <topic>`
- `pulsarctl topics set-inactive-topic-policies <topic> --enable-delete-while-inactive <bool> --max-inactive-duration <string> --delete-mode <delete_when_no_subscriptions|delete_when_subscriptions_caught_up>`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants