Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PIP-82] [pulsar-broker] incorporate review feedback #10201

Merged
merged 1 commit into from
Apr 17, 2021

Conversation

bharanic-dev
Copy link
Contributor

Motivation

Follow up and incorporate the review feedback.

Modifications

  • Have lightproto generate sources to a different directory so it doesn't come in the way of protobuf generated sources.
  • make resourceUsageTransportPublishTopicName internal(remove from config)
  • drop old/stale resource-usage messages
  • fix a race condition in creating tenant/namespace for resource-usage when multiple brokers attempt to do it.

Verifying this change

  • Make sure that the change passes the CI checks.

This change is a trivial rework / code cleanup without any test coverage.

@@ -687,12 +687,6 @@
)
private String resourceUsageTransportClassName = "";

@FieldContext(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the rationale for making this not configurable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jerrypeng while reviewing the previous PR(#10008), @codelipenghui and @315157973 gave feedback that the topic name is internal implementation detail that the user doesn't need to be exposed to. I did not have a strong reason to go the other way, so made this change. We can always make it configurable in the future, if we find that there is a use-case for it.

@@ -159,9 +171,11 @@ public void received(Reader<byte[]> reader, Message<byte[]> msg) {
private final Map<String, ResourceUsageConsumer>
consumerMap = new ConcurrentHashMap<String, ResourceUsageConsumer>();

private long staleMessageCount = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This variable doesn't seem to be used anywhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added the variable with the intention of adding a unit test for it. But could not figure out a easy way to do so. I am open to ideas if you have suggestions on how I can add one. Else, I will remove this variable.

@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@sijie sijie added this to the 2.8.0 milestone Apr 13, 2021
@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

5 similar comments
@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

- Have lightproto generate sources to a different directory so it doesn't come in the way of protobuf generated sources.
- make resourceUsageTransportPublishTopicName internal(remove from config)
- drop old/stale resource-usage messages
- fix a race condition in creating tenant/namespace for resource-usage when multiple brokers attempt to do it.
@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

2 similar comments
@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@bharanic-dev
Copy link
Contributor Author

/pulsarbot run-failure-checks

@jerrypeng jerrypeng merged commit 1cc782f into apache:master Apr 17, 2021
@linlinnn
Copy link
Contributor

@jerrypeng CICD seems broken for this merge

@linlinnn
Copy link
Contributor

image

@linlinnn
Copy link
Contributor

linlinnn commented Apr 17, 2021

Seems should remove this line
this.conf.setResourceUsageTransportPublishTopicName(INTERNAL_TOPIC);
but actually remove this.conf.setResourceUsageTransportClassName("org.apache.pulsar.broker.resourcegroup.ResourceUsageTransportManager");

Same in ResourceGroupUsageAggregationTest

linlinnn added a commit to linlinnn/pulsar that referenced this pull request Apr 17, 2021
@linlinnn
Copy link
Contributor

@merlimat PTAL

lhotari added a commit to lhotari/pulsar that referenced this pull request Apr 17, 2021
@lhotari
Copy link
Member

lhotari commented Apr 17, 2021

@linlinnn @jerrypeng @codelipenghui @eolivelli @merlimat @aahmed-se I created #10250 to fix the compilation issues in master branch.

aahmed-se pushed a commit that referenced this pull request Apr 17, 2021
@@ -172,12 +186,26 @@ private void createTenantAndNamespace() throws PulsarServerException, PulsarAdmi

List<String> tenantList = admin.tenants().getTenants();
if (!tenantList.contains(tenant)) {
admin.tenants().createTenant(tenant,
new TenantInfo(Sets.newHashSet(config.getSuperUserRoles()), Sets.newHashSet(cluster)));
try {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To add try/catch is a good solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants