Skip to content

Comments

[bugfix] Unavailable instance should not be added to instanceToSegmentsMap in strictReplicaGroup#10302

Closed
61yao wants to merge 4 commits intoapache:masterfrom
61yao:debug_unavailable_assign
Closed

[bugfix] Unavailable instance should not be added to instanceToSegmentsMap in strictReplicaGroup#10302
61yao wants to merge 4 commits intoapache:masterfrom
61yao:debug_unavailable_assign

Conversation

@61yao
Copy link
Contributor

@61yao 61yao commented Feb 17, 2023

No description provided.

@61yao 61yao marked this pull request as draft February 17, 2023 22:05
@Nullable
private List<String> calculateEnabledInstancesForSegment(String segment, List<String> onlineInstancesForSegment,
Set<String> unavailableSegments) {
Set<String> unavailableSegments, String debugString) {
Copy link
Contributor

@snleee snleee Feb 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we avoid addingdebugString? Instead, we can log whether it's from assignment/instance change from the caller. Then, we should be able to check the logs to find the same information.

LOGGER.info("Calculating enabled instances on assignment change");
List<String> enabledInstancesForSegment =
          calculateEnabledInstancesForSegment(segment, entry.getValue(), unavailableSegments);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I do log info, we will have too much logs since we log one entry even when there is no error?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh this is getting called within the loop. You can add the log before the loop starts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is also a lot of logging because we don't need the log when enabledInstancesForSegment is not empty?

onlineInstances.add(instance);
instanceToSegmentsMap.computeIfAbsent(instance, k -> new ArrayList<>()).add(segment);
}
if (!onlineInstances.isEmpty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add the TODO comment about ignoring for the new segments only?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@codecov-commenter
Copy link

codecov-commenter commented Feb 18, 2023

Codecov Report

Merging #10302 (171641b) into master (a016255) will increase coverage by 0.05%.
The diff coverage is 91.66%.

@@             Coverage Diff              @@
##             master   #10302      +/-   ##
============================================
+ Coverage     70.36%   70.42%   +0.05%     
+ Complexity     5827     5825       -2     
============================================
  Files          2016     2016              
  Lines        109635   109651      +16     
  Branches      16680    16682       +2     
============================================
+ Hits          77145    77218      +73     
+ Misses        27073    27011      -62     
- Partials       5417     5422       +5     
Flag Coverage Δ
integration1 24.60% <25.00%> (+0.14%) ⬆️
integration2 24.52% <83.33%> (+<0.01%) ⬆️
unittests1 67.68% <ø> (-0.03%) ⬇️
unittests2 13.70% <91.66%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ceselector/StrictReplicaGroupInstanceSelector.java 93.65% <88.88%> (-1.81%) ⬇️
...routing/instanceselector/BaseInstanceSelector.java 99.10% <100.00%> (ø)
...nt/local/startree/v2/store/StarTreeDataSource.java 37.50% <0.00%> (-18.75%) ⬇️
...nction/DistinctCountBitmapAggregationFunction.java 47.66% <0.00%> (-8.30%) ⬇️
...or/transform/function/IsNullTransformFunction.java 79.31% <0.00%> (-6.90%) ⬇️
...transform/function/IsNotNullTransformFunction.java 65.51% <0.00%> (-6.90%) ⬇️
...ot/segment/local/startree/OffHeapStarTreeNode.java 65.90% <0.00%> (-6.82%) ⬇️
...l/segment/index/readers/OnHeapFloatDictionary.java 86.36% <0.00%> (-4.55%) ⬇️
...core/startree/operator/StarTreeFilterOperator.java 88.15% <0.00%> (-3.29%) ⬇️
.../java/org/apache/pinot/spi/data/TimeFieldSpec.java 88.63% <0.00%> (-2.28%) ⬇️
... and 34 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@61yao 61yao marked this pull request as ready for review February 18, 2023 00:33
@61yao 61yao requested a review from snleee February 18, 2023 00:44
@61yao 61yao closed this Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants