Fix defect ensemble model (with Code) and genetic algorithm #2317

alfoa · 2024-05-08T03:16:02Z

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

What are the significant changes in functionality due to this change request?

The EnsembleModel (with code) uses a different parallelization strategy. The batching mode has been enabled in that strategy (parallelMode ==2)

I also created an issue #2318 to better re-design the batching system

For Change Control Board: Change Request Review

The following review must be completed by an authorized member of the Change Control Board.

1. Review all computer code.
2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
5. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options. Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in XML block, the node <internalParallel> to True.
6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change. The requirements and the requirements test shall be in sync.
7. The merge request must reference an issue. If the issue is closed, the issue close checklist shall be done.
8. If an analytic test is changed/added is the the analytic documentation updated/added?
9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

.../GeneticAlgorithms/continuous/unconstrained/test_ensembleModel_withGA_Code_and_Functions.xml

…trained/test_ensembleModel_withGA_Code_and_Functions.xml

…ven into alfoa/genetic_ensemble_model

...lgorithms/continuous/unconstrained/metaModelWithCodeAndFunctionsAndGenetic/decayConstantB.py

…trained/metaModelWithCodeAndFunctionsAndGenetic/decayConstantB.py

wangcj05

Just one question, changes are good to me.

wangcj05 · 2024-05-08T16:11:04Z

.../GeneticAlgorithms/continuous/unconstrained/test_ensembleModel_withGA_Code_and_Functions.xml

+        optimize
+    </Sequence>
+    <WorkingDir>metaModelWithCodeAndFunctionsAndGenetic</WorkingDir>
+    <batchSize>1</batchSize>


Have you tested it with different batch size?

ops. Not yet. let me test it

@wangcj05 I fixed the parallel execution and converted the test into a parallel (batchSize > 1) to test it.

Ready for re-review

wangcj05

The changes are good. @alfoa Could you document how you define job '' prefix'' and 'uniqueHandler' inside Ensemble? This will help for future rework on the job submission function.

wangcj05 · 2024-05-09T19:22:10Z

tests/framework/Optimizers/GeneticAlgorithms/tests

+    type = 'RavenFramework'
+    input = 'continuous/unconstrained/test_ensembleModel_withGA_Code_and_Functions.xml'
+    [./csv]
+     type = UnorderedCsv


@alfoa FYI, you need to change 'UnorderedCsv' to 'UnorderedCSV'. There is an inconsistent naming in the rook test system.

done and added an error msg in ROOK (to avoid a blind crash)

moosebuild · 2024-05-09T19:44:56Z

Job Test mac on 54f43a4 : invalidated by @wangcj05

RuntimeError: Found incorrect download: libaec. Aborting

alfoa · 2024-05-09T20:17:16Z

The changes are good. @alfoa Could you document how you define job '' prefix'' and 'uniqueHandler' inside Ensemble? This will help for future rework on the job submission function.

I can add the info in the Issue I created?

moosebuild · 2024-05-10T18:57:28Z

Job Test Ubuntu 20-2 Optional on a522493 : invalidated by @wangcj05

failed at fetch

moosebuild · 2024-05-13T14:26:29Z

Job Test qsubs sawtooth on a522493 : invalidated by @alfoa

Set python environment taking 6+ hrs?

alfoa · 2024-05-13T18:03:53Z

Job Test qsubs sawtooth on a522493 : invalidated by @alfoa

Set python environment taking 6+ hrs?

@wangcj05 There is a problem with HPC testing (set up enviroment) that is not related to this PR. (It does not seem to be a spurious problem since I invalidated the job multiple times and it gets stuck at the same step)

alfoa · 2024-05-13T18:11:37Z

@wangcj05 @joshua-cogliati-inl It seems that now in devel/main there is a cascade of failures (https://civet.inl.gov/branch/2903/) after merge of PR #2309 (I dnk if it is related...probably it is not)

rook/main.py

alfoa · 2024-05-13T22:17:11Z

@wangcj05 now it fails because cvxpy is missing?

…exceptions

alfoa · 2024-05-13T23:05:13Z

scripts/library_handler.py

@@ -54,7 +54,7 @@
 # some bad actors can't use the metadata correctly
 # and so need special treatment
 # -> see findLibAndVersion
-metaExceptions = ['pyside2', 'AMSC', 'PIL']
+metaExceptions = ['pyside2', 'AMSC', 'PIL', 'cvxpy']


'cvxpy' is added via FARM plugin. It cannot be checked with the metadata. I followed the approach in the library_handler but this should be revised since it is not easily maintainable (overall for plugins). @mandd @wangcj05 @joshua-cogliati-inl . I wonder why the regression system (for PR) did not catch this.

It seems this is due to an update of the library in conda-forge 2 days ago (see https://anaconda.org/conda-forge/cvxpy-base/files).

'cvxpy' is added via FARM plugin. It cannot be checked with the metadata. I followed the approach in the library_handler but this should be revised since it is not easily maintainable (overall for plugins). @mandd @wangcj05 @joshua-cogliati-inl . I wonder why the regression system (for PR) did not catch this.

I changed the approach..if a library through a metadata is not found, we try the "slow approach".

skip_check is another way to fix this. Example: <hdf5 skip_check='True'/> I have no objection to fixing it in metaExceptions however.

moosebuild · 2024-05-14T13:28:33Z

Job Mingw Test on 7ff7659 : invalidated by @alfoa

alfoa · 2024-05-14T15:54:08Z

@wangcj05 the Mingw Test are failing and I don't see a correlation with this MR.

moosebuild · 2024-05-14T17:52:01Z

Job Mingw Test on 7ff7659 : invalidated by @alfoa

joshua-cogliati-inl · 2024-05-14T19:14:27Z

@wangcj05 the Mingw Test are failing and I don't see a correlation with this MR.

How many tests are failing? (And if only a few, which ones?) Thanks.

wangcj05 · 2024-05-14T19:31:36Z

@wangcj05 the Mingw Test are failing and I don't see a correlation with this MR.

How many tests are failing? (And if only a few, which ones?) Thanks.

FYI, there are around 33 tests failed in previous MingW test. Many for optimizations, such as simulated annealing. I have checked the new tests right now, it seems the runs are ok for now. Let's see if the test on Mingw can pass or not. @joshua-cogliati-inl

alfoa · 2024-05-14T19:52:43Z

@wangcj05 the Mingw Test are failing and I don't see a correlation with this MR.

How many tests are failing? (And if only a few, which ones?) Thanks.

FYI, there are around 33 tests failed in previous MingW test. Many for optimizations, such as simulated annealing. I have checked the new tests right now, it seems the runs are ok for now. Let's see if the test on Mingw can pass or not. @joshua-cogliati-inl

@wangcj05 @joshua-cogliati-inl unfortunately they are still failing...now in the TSA, PostProcessors, Datamining etc.

wangcj05 · 2024-05-14T19:56:30Z

@joshua-cogliati-inl Let's try to test it on devel first. Can you pull the changes for cvxpy and test it on your trivial white space branch?

joshua-cogliati-inl · 2024-05-14T20:19:26Z

@joshua-cogliati-inl Let's try to test it on devel first. Can you pull the changes for cvxpy and test it on your trivial white space branch?

Hm, which set of changes for cvxpy? d3c7069 ?

alfoa · 2024-05-14T21:02:50Z

@wangcj05 the Mingw Test are failing and I don't see a correlation with this MR.

How many tests are failing? (And if only a few, which ones?) Thanks.

FAILED:

Timeout tests\framework\PostProcessors\SubdomainBasicStatistics\subdomainSensitivity

Timeout tests\framework\PostProcessors\SubdomainBasicStatistics\subdomainTimeDepStats

Failed tests\framework\PostProcessors\TemporalDataMiningPostProcessor\Clustering\KMeans

Failed tests\framework\PostProcessors\TemporalDataMiningPostProcessor\Clustering\MiniBatchKMeans

Failed tests\framework\PostProcessors\TemporalDataMiningPostProcessor\Clustering\DBSCAN

Failed tests\framework\PostProcessors\TemporalDataMiningPostProcessor\Clustering\MeanShift

Failed tests\framework\PostProcessors\TemporalDataMiningPostProcessor\Clustering\AffinityPropogation

Failed tests\framework\PostProcessors\TemporalDataMiningPostProcessor\Clustering\SpectralClustering

Failed tests\framework\PostProcessors\TSACharacterizer\Basic

Failed tests\framework\PostProcessors\TopologicalPostProcessor\knn

Timeout tests\framework\PostProcessors\TopologicalPostProcessor\relaxed_beta_skeleton

Failed tests\framework\PostProcessors\TopologicalPostProcessor\beta_skeleton

Failed tests\framework\PostProcessors\TSACharacterizer\RWD

Failed tests\framework\PostProcessors\Validation\test_validation_gate_probabilistic_time_dep

Failed tests\framework\PostProcessors\Validation\test_validation_gate_probabilistic

Timeout tests\framework\PostProcessors\Validation\test_validation_gate_pcm

Diff tests\framework\ROM\FeatureSelection\DMDcRFEScoringSubgroup

Failed tests\framework\ROM\FeatureSelection\DMDcRFEScoringOnlyOutput

Failed tests\framework\ROM\FeatureSelection\DMDcRFEApplyClustering

Failed tests\framework\ROM\FeatureSelection\DMDcRFESubgroupCrossCorrelation

Timeout tests\framework\ROM\FeatureSelection\StaticSklearnVarianceThreshold

Failed tests\framework\ROM\FeatureSelection\DMDcVarianceThreshold

Failed tests\framework\ROM\MSR\cosine

Timeout tests\framework\ROM\MSR\biweight

Failed tests\framework\ROM\MSR\Epanechnikov

Failed tests\framework\ROM\MSR\exponential

Timeout tests\framework\ROM\MSR\logistic

Timeout tests\framework\ROM\MSR\Gaussian

Timeout tests\framework\ROM\MSR\Silverman

Failed tests\framework\ROM\MSR\triangular

Failed tests\framework\ROM\MSR\tricube

Failed tests\framework\ROM\MSR\triweight

Timeout tests\framework\ROM\MSR\uniform

alfoa · 2024-05-14T21:04:00Z

@joshua-cogliati-inl Let's try to test it on devel first. Can you pull the changes for cvxpy and test it on your trivial white space branch?

Hm, which set of changes for cvxpy? d3c7069 ?

7ff7659

joshua-cogliati-inl · 2024-05-14T21:22:12Z

@joshua-cogliati-inl Let's try to test it on devel first. Can you pull the changes for cvxpy and test it on your trivial white space branch?

Hm, which set of changes for cvxpy? d3c7069 ?

7ff7659

Thanks: #2113

moosebuild · 2024-05-15T21:10:27Z

Job Mingw Test on 7ff7659 : invalidated by @alfoa

wangcj05

changes are good.

wangcj05 · 2024-05-16T04:43:32Z

Changes are good, PR checklist is good. PR can be merged.

alfoa added 2 commits May 7, 2024 21:14

Closes #2304

a0ea3b5

added test

e9feed6

alfoa commented May 8, 2024

View reviewed changes

.../GeneticAlgorithms/continuous/unconstrained/test_ensembleModel_withGA_Code_and_Functions.xml Outdated Show resolved Hide resolved

alfoa added 3 commits May 7, 2024 21:54

Update tests/framework/Optimizers/GeneticAlgorithms/continuous/uncons…

336bae3

…trained/test_ensembleModel_withGA_Code_and_Functions.xml

removed trailing

da32086

Merge branch 'alfoa/genetic_ensemble_model' of github.com:idaholab/ra…

70a5354

…ven into alfoa/genetic_ensemble_model

alfoa requested review from wangcj05, mandd and Jimmy-INL May 8, 2024 03:58

alfoa added the Ready To Review label May 8, 2024

added missing input file

6077f56

alfoa commented May 8, 2024

View reviewed changes

...lgorithms/continuous/unconstrained/metaModelWithCodeAndFunctionsAndGenetic/decayConstantB.py Outdated Show resolved Hide resolved

Update tests/framework/Optimizers/GeneticAlgorithms/continuous/uncons…

c7f27bf

…trained/metaModelWithCodeAndFunctionsAndGenetic/decayConstantB.py

mandd previously approved these changes May 8, 2024

View reviewed changes

wangcj05 reviewed May 8, 2024

View reviewed changes

alfoa added 2 commits May 9, 2024 07:24

isinstance

af23869

fixed parallel execution

ee2f53f

alfoa dismissed mandd’s stale review via ee2f53f May 9, 2024 18:36

alfoa requested review from wangcj05 and mandd May 9, 2024 18:37

alfoa added 2 commits May 9, 2024 12:54

readded heron and baycal?

7fba807

removed trailing spaces

9f077b8

alfoa mentioned this pull request May 9, 2024

[TASK] Developers: Batching system re-design #2318

Open

10 tasks

some cleaning

ab60b73

wangcj05 reviewed May 9, 2024

View reviewed changes

fixed test name and rook error message

54f43a4

alfoa closed this May 9, 2024

alfoa commented May 13, 2024

View reviewed changes

rook/main.py Outdated Show resolved Hide resolved

alfoa added 2 commits May 13, 2024 15:42

Apply suggestions from code review

5856f6b

removed trailing whitespaces

f83fb30

alfoa added 2 commits May 13, 2024 16:23

Merge branch 'devel' into alfoa/genetic_ensemble_model

91a23d9

fix for 'cvxpy'...this is not a very maintainable way to handle such …

d3c7069

…exceptions

alfoa commented May 13, 2024

View reviewed changes

changed approach

7ff7659

mandd approved these changes May 14, 2024

View reviewed changes

alfoa mentioned this pull request May 14, 2024

SimulatedAnnealing Discrete Variables #2312

Merged

9 tasks

wangcj05 approved these changes May 16, 2024

View reviewed changes

wangcj05 merged commit 782ccc0 into devel May 16, 2024
12 checks passed

wangcj05 deleted the alfoa/genetic_ensemble_model branch May 16, 2024 04:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix defect ensemble model (with Code) and genetic algorithm #2317

Fix defect ensemble model (with Code) and genetic algorithm #2317

alfoa commented May 8, 2024 •

edited by wangcj05

Loading

wangcj05 left a comment

wangcj05 May 8, 2024

alfoa May 8, 2024

alfoa May 9, 2024

wangcj05 left a comment

wangcj05 May 9, 2024

alfoa May 9, 2024

moosebuild commented May 9, 2024

alfoa commented May 9, 2024 •

edited

Loading

moosebuild commented May 10, 2024

moosebuild commented May 13, 2024

alfoa commented May 13, 2024

alfoa commented May 13, 2024

alfoa commented May 13, 2024

alfoa May 13, 2024 •

edited

Loading

alfoa May 13, 2024

alfoa May 13, 2024

joshua-cogliati-inl May 14, 2024

moosebuild commented May 14, 2024

alfoa commented May 14, 2024

moosebuild commented May 14, 2024

joshua-cogliati-inl commented May 14, 2024

wangcj05 commented May 14, 2024

alfoa commented May 14, 2024

wangcj05 commented May 14, 2024

joshua-cogliati-inl commented May 14, 2024

alfoa commented May 14, 2024

alfoa commented May 14, 2024

joshua-cogliati-inl commented May 14, 2024

moosebuild commented May 15, 2024

wangcj05 left a comment

wangcj05 commented May 16, 2024

Fix defect ensemble model (with Code) and genetic algorithm #2317

Fix defect ensemble model (with Code) and genetic algorithm #2317

Conversation

alfoa commented May 8, 2024 • edited by wangcj05 Loading

Pull Request Description

What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)

What are the significant changes in functionality due to this change request?

For Change Control Board: Change Request Review

wangcj05 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wangcj05 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moosebuild commented May 9, 2024

alfoa commented May 9, 2024 • edited Loading

moosebuild commented May 10, 2024

moosebuild commented May 13, 2024

alfoa commented May 13, 2024

alfoa commented May 13, 2024

alfoa commented May 13, 2024

alfoa May 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

moosebuild commented May 14, 2024

alfoa commented May 14, 2024

moosebuild commented May 14, 2024

joshua-cogliati-inl commented May 14, 2024

wangcj05 commented May 14, 2024

alfoa commented May 14, 2024

wangcj05 commented May 14, 2024

joshua-cogliati-inl commented May 14, 2024

alfoa commented May 14, 2024

alfoa commented May 14, 2024

joshua-cogliati-inl commented May 14, 2024

moosebuild commented May 15, 2024

wangcj05 left a comment

Choose a reason for hiding this comment

wangcj05 commented May 16, 2024

alfoa commented May 8, 2024 •

edited by wangcj05

Loading

alfoa commented May 9, 2024 •

edited

Loading

alfoa May 13, 2024 •

edited

Loading