Skip to content

[STORM-3282] Fix RAS worker count estimation#2902

Merged
asfgit merged 2 commits intoapache:masterfrom
kishorvpatil:storm3282
Nov 8, 2018
Merged

[STORM-3282] Fix RAS worker count estimation#2902
asfgit merged 2 commits intoapache:masterfrom
kishorvpatil:storm3282

Conversation

@kishorvpatil
Copy link
Copy Markdown
Contributor

No description provided.

Copy link
Copy Markdown
Contributor

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@kishorvpatil
Copy link
Copy Markdown
Contributor Author

The jenkins failure seems unrelated.

Copy link
Copy Markdown
Contributor

@Ethanlm Ethanlm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry my bad. Didn't know we have TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB.

return (int) Math.ceil(getEstimatedTotalHeapMemoryRequiredByTopo(topoConf, topology) /
ObjectReader.getDouble(topoConf.get(Config.WORKER_HEAP_MEMORY_MB)));
Double defaultWorkerMaxHeap = ObjectReader.getDouble(topoConf.get(Config.WORKER_HEAP_MEMORY_MB));
Double topologyWorkerMaxHeap = ObjectReader.getDouble(topoConf.get(Config.TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will topoConf.get(Config.WORKER_HEAP_MEMORY_MB) be null? If so, this will throw exception IllegalArgumentException("Don't know how to convert null to double");

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is default of this value - https://github.com/apache/storm/blob/master/conf/defaults.yaml#L332 and https://github.com/apache/storm/blob/master/conf/defaults.yaml#L332 avoids it. But we can
So I don't anticipate that happening, but I am adding defaults.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. ok. If there is a default value in yaml file, then that's fine. I think we don't want to have to maintain two default values both in default.yaml and also in the code.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default in the code is last resort, as will be picked if we ever remove defaults from yaml

Copy link
Copy Markdown
Contributor

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@asfgit asfgit merged commit 2ba69e2 into apache:master Nov 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants