Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distributed Mode for SamplerPostprocessorTransfer and StochasticResults #14410

Closed
aeslaughter opened this issue Nov 21, 2019 · 4 comments · Fixed by #14437
Closed

Distributed Mode for SamplerPostprocessorTransfer and StochasticResults #14410

aeslaughter opened this issue Nov 21, 2019 · 4 comments · Fixed by #14437
Assignees
Labels
C: Modules P: normal A defect affecting operation with a low possibility of significantly affects. T: task An enhancement to the software.

Comments

@aeslaughter
Copy link
Contributor

aeslaughter commented Nov 21, 2019

Reason

The aim is to be capable of handling over 1e9 stochastic runs. With 1e9 runs a single PP will require 8GB to store on a single rank. Thus, the resulting data must have the option to remain distributed.

Design

The SamplerPostprocessorTransfer passes data to StochasticResult VPP, currently the transfer is doing parallel communication to build the complete data that is sent to the VPP. This should only communicate the local part.

The StochasticResult object is a VPP designed to hold a vectors of PP values from the sub-application runs. This object should have a "parallel_type" option that control if the data is distributed or replicated across ranks. When populated by a transfer it should only have the local part, the "parallel_type" should dictate if the communication should happen.

Also, if "parallel_type = distributed" then the StochasticResult should warn/error if CSV output is enabled.

Impact

This will change will allow for very large stochastic runs to distribute the stochastic simulation data for use by other aggregation routines, such as confidence level intervals, to reduce the data reported to a manageable size.

@aeslaughter aeslaughter added T: task An enhancement to the software. P: normal A defect affecting operation with a low possibility of significantly affects. labels Nov 21, 2019
@aeslaughter
Copy link
Contributor Author

In distributed mode, there should also be an option to output values from a specified processor. This will mainly be for testing.

@aeslaughter aeslaughter self-assigned this Nov 21, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 21, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 21, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 21, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 25, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 25, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 25, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 25, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 25, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Nov 26, 2019
@jiangwen84
Copy link
Contributor

jiangwen84 commented Dec 3, 2019

If we cannot output the StochasticResults into a CSV, how do we analyze the statistical results?

@aeslaughter
Copy link
Contributor Author

Handling the output is future work: see #14480

aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 5, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 10, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 11, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 11, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 18, 2019
aeslaughter added a commit to aeslaughter/moose that referenced this issue Dec 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: Modules P: normal A defect affecting operation with a low possibility of significantly affects. T: task An enhancement to the software.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants