Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue 11493] Fix #11493. Simple implementation of getting number of references from C++ client #11535

Merged

Conversation

Sunny-Island
Copy link
Contributor

Fixes #11493

Master Issue: #11493

Motivation

In Pulsar, we use a single client to create multiple producers/consumers/readers. Is there any method/attribute that can give information on number of producers/readers/consumers connected to the given pulsar client instance at the given point of time?

Say there is a single pulsar client instance. Multiple consumers and readers are created from the given client instance. This client needs to be cleaned up when all the references are closed. In this case, it would be of help, to get information on the number of the consumers/readers/ getPartitionsForTopic calls are active on the given client. Ie, having the number of references for the given client can provide information on whether it is fine to clean up the client instance at the given point of time.

Explain here the context, and why you're making that change. What is the problem you're trying to solve.

Modifications

Add these method to Client.h, Client.cc, ClientImpl.h, CliemtImpl.cc :

uint64_t getNumberOfProducer()
uint64_t getNumberOfConsumer()
uint64_t getNumberOfReader() (May not be used)

To count alive producers, I get each producer by weak point and check if it is connected.

Verifying this change

This change added tests and can be verified as follows:

Add unit test to check if these functions can return correct references number. Test file is ClientTest.cc. Test function is TEST(ClientTest, testGetNumberOfReferences).

Documentation

For this PR, do we need to update docs?

I will add doc to C++ client part in follow-up issue.

@Sunny-Island
Copy link
Contributor Author

Sunny-Island commented Aug 2, 2021

There are two questions:

  1. For PartitionedProducer/PartitionedConsumer/MultiTopicConsumer which consist of more than one consumer/producer. They should be counted as one or more reference?
  2. When created reader from client, client add reader's consumer to its consumer list. Should we remove getNumberOfReader()? I haven't implemented this function yet.

@BewareMyPower
Copy link
Contributor

For PartitionedProducer/PartitionedConsumer/MultiTopicConsumer which consist of more than one consumer/producer. They should be counted as one or more reference?

I think it should be counted as multiple references. For example, if you created a producer of a 3 partitions topic, there're actually 3 TCP connections.

When created reader from client, client add reader's consumer to its consumer list. Should we remove getNumberOfReader()? I haven't implemented this function yet.

Reader is just treated as a Consumer, so I think we can remove the getNumberOfReader() method.

pulsar-client-cpp/lib/ClientImpl.cc Outdated Show resolved Hide resolved
@Anonymitaet Anonymitaet added the doc-required Your PR changes impact docs and you will update later. label Aug 3, 2021
@Anonymitaet
Copy link
Member

@Sunny-Island you can take this as references when adding docs.

@BewareMyPower BewareMyPower added doc-not-needed Your PR changes do not impact docs and removed doc-required Your PR changes impact docs and you will update later. labels Aug 3, 2021
@BewareMyPower
Copy link
Contributor

@Anonymitaet I changed the label from doc-required to no-need-doc because the documents are already added to the C++ header so that the related documents will be generated by Doxygen.

@Anonymitaet
Copy link
Member

@BewareMyPower many thanks for providing doc info! Since this PR modifies docs (it adds some descriptions in .h files and they are "docs"), I label this PR with doc.

pulsar-client-cpp/include/pulsar/Client.h Outdated Show resolved Hide resolved
pulsar-client-cpp/include/pulsar/Client.h Outdated Show resolved Hide resolved
pulsar-client-cpp/include/pulsar/Client.h Outdated Show resolved Hide resolved
@Anonymitaet Anonymitaet added doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. and removed doc-not-needed Your PR changes do not impact docs labels Aug 3, 2021
@BewareMyPower BewareMyPower merged commit e6909c6 into apache:master Aug 3, 2021
@codelipenghui codelipenghui added this to the 2.9.0 milestone Aug 4, 2021
codelipenghui pushed a commit that referenced this pull request Aug 5, 2021
…rom C++ client (#11535)

Fixes #11493 

Master Issue: #11493 

### Motivation
In Pulsar, we use a single client to create multiple producers/consumers/readers. Is there any method/attribute that can give information on number of producers/readers/consumers connected to the given pulsar client instance at the given point of time?

Say there is a single pulsar client instance. Multiple consumers and readers are created from the given client instance. This client needs to be cleaned up when all the references are closed. In this case, it would be of help, to get information on the number of the consumers/readers/ getPartitionsForTopic calls are active on the given client. Ie, having the number of references for the given client can provide information on whether it is fine to clean up the client instance at the given point of time.

*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*

### Modifications

Add these method to `Client.h`, `Client.cc`, `ClientImpl.h`, `CliemtImpl.cc` :
```
uint64_t getNumberOfProducer()
uint64_t getNumberOfConsumer()
```
To count alive producers, I get each producer by weak point and check if it is connected.

### Verifying this change

This change added tests and can be verified as follows:

Add unit test to check if these functions can return correct references number. Test file is `ClientTest.cc`. Test function is `TEST(ClientTest, testGetNumberOfReferences)`.

(cherry picked from commit e6909c6)
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Aug 5, 2021
LeBW pushed a commit to LeBW/pulsar that referenced this pull request Aug 9, 2021
…rom C++ client (apache#11535)

Fixes apache#11493 

Master Issue: apache#11493 

### Motivation
In Pulsar, we use a single client to create multiple producers/consumers/readers. Is there any method/attribute that can give information on number of producers/readers/consumers connected to the given pulsar client instance at the given point of time?

Say there is a single pulsar client instance. Multiple consumers and readers are created from the given client instance. This client needs to be cleaned up when all the references are closed. In this case, it would be of help, to get information on the number of the consumers/readers/ getPartitionsForTopic calls are active on the given client. Ie, having the number of references for the given client can provide information on whether it is fine to clean up the client instance at the given point of time.

*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*

### Modifications

Add these method to `Client.h`, `Client.cc`, `ClientImpl.h`, `CliemtImpl.cc` :
```
uint64_t getNumberOfProducer()
uint64_t getNumberOfConsumer()
```
To count alive producers, I get each producer by weak point and check if it is connected.

### Verifying this change

This change added tests and can be verified as follows:

Add unit test to check if these functions can return correct references number. Test file is `ClientTest.cc`. Test function is `TEST(ClientTest, testGetNumberOfReferences)`.
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
…rom C++ client (apache#11535)

Fixes apache#11493 

Master Issue: apache#11493 

### Motivation
In Pulsar, we use a single client to create multiple producers/consumers/readers. Is there any method/attribute that can give information on number of producers/readers/consumers connected to the given pulsar client instance at the given point of time?

Say there is a single pulsar client instance. Multiple consumers and readers are created from the given client instance. This client needs to be cleaned up when all the references are closed. In this case, it would be of help, to get information on the number of the consumers/readers/ getPartitionsForTopic calls are active on the given client. Ie, having the number of references for the given client can provide information on whether it is fine to clean up the client instance at the given point of time.

*Explain here the context, and why you're making that change. What is the problem you're trying to solve.*

### Modifications

Add these method to `Client.h`, `Client.cc`, `ClientImpl.h`, `CliemtImpl.cc` :
```
uint64_t getNumberOfProducer()
uint64_t getNumberOfConsumer()
```
To count alive producers, I get each producer by weak point and check if it is connected.

### Verifying this change

This change added tests and can be verified as follows:

Add unit test to check if these functions can return correct references number. Test file is `ClientTest.cc`. Test function is `TEST(ClientTest, testGetNumberOfReferences)`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/client cherry-picked/branch-2.8 Archived: 2.8 is end of life doc Your PR contains doc changes, no matter whether the changes are in markdown or code files. release/2.8.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Get the number of references to the given pulsar client instance at a given point of time.
6 participants