Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Support SSMT in Idmtools (InstituteforDiseaseModeling#587)
* Fix InstituteforDiseaseModeling#585 and initial platform check-in for SSMT * Add ssmt examples * SSMT: update idmtools.ini * SSMT: add utility files * SSMT: add examples related files * SSMT: update examples * SSMT: update examples using IAnalyzer as class parent * SSMT: code clean up a bit * SSMT: implemented SSMTWorkItem and WorkItemManager * SSMT: implemented work item operations * SSMT: implemented required changes on iplatform, iplatform_metadata and enums * SSMT: update custom analyzers * SSMT: add len for FileList * SSMT: make sure asset_files working (update SSMT operations) * SSMT: fix existing Suite type checking * SSMT: refactor code: use COMPSPlatform (instead of SSMTPlatform) to work with SSMT * SSMT: refactor code: code cleanup * SSMT: refactor code: cleanup code a bit * SSMT: update three samples to use COMPSPlatform instead of SSMTPlatform * SSMT: add a new example with Assets * ssmtanalysis test multiple args * SSMT: SSMTAnalysis now takes platform as one parameter so that it can use different configurations * SSMT: SSMTAnalysis automatically upload idmtools.ini * add tests for ssmt work item scenarios * SSMT: feature implementation: WorkItem now can have its own docker image * SSMT: initial implementation: retrieve WorkItem files * update for WI docker_image change * SSMT: refactor LocalOS to make it simpler * SSMT: implementation of retrieving files from WorkItem * SSMT: removed ItemType.WorkItem checking * add members to COMPSPlatform: max_workers and batch_size * SSMT: update idmtools.ini * additional ssmt and wi tests * SSMT: fix local_os WINDOWS checking issue * SSMT: removed IPlatformWorkItemOperations * SSMT: add is_task_supported back (removed by mistake) * SSMT: make output folder only available to temType.SIMULATION and ItemType.WORKFLOW_ITEM * placeholder vistools test * SSMT: cast uuid to str * modify test for ssmt analyzer for multiple experiments * SSMT: track work item created by SSMTAnalysis * add run_ssmt_analysis script and fix del_file util ref * pass exp list * SSMT: add method to retrieve created work item * SSMT: SSMTWorkItem support InputData and vistools and more related object * SSMT: iplatform implemented get_related_items method * Fix InstituteforDiseaseModeling#596: make sure COMPSPlatform.get_files working for Experiment * Fix InstituteforDiseaseModeling#596: temp workarounds * SSMT: initial implementation WorkItem Operations to support InputData and vistool and related objects * SSMT: add example for InputDataWorker type * SSMT: add test files * updating to use idmtools workitem support * SSMT: add vistools sample * SSMT: re-organized WorkItem examples folder structure * SSMT: delete my test file * add inputdata work item tests * SSMT: re-organized workitem examples * uncomment unittest import * SSMT: fix import issue due to folder structure changes * clean up SSMT work item tests and typos * Fix InstituteforDiseaseModeling#598 * Adding bin dir for emod example which missing in dev branch * ssmt tests for python analysis non emod * remove local analysis file * SSMT: improve code a little bit * SSMT: improve IPlatform definition with dataclass field * SSMT: improve IPlatform definition with dataclass field and also improved get_files_by_id method * SSMT: code cleanup a bit * SSMT: re-named an example name from simple tp simple_assets * SSMT: re-named file name FileList.py to file_list.py * update vistools integtation test and add real example * add missing bin folder * remove kurt's example and add another general example * SSMT: initial refactor SSMTWorkItem (updated tests as well) * SSMT: refactor SSMTWorkItem: re-name files * SSMT: add missing files back * SSMT: relocate files * SSMT: relocate files * SSMT: update example: removed test parameter * SSMT: delete un-used data * SSMT: fix analyzer command issue * Add/update all ssmt integration tests * fix csvanalyzer test issue in test_analyzers_emod_comps.py and rename vistools test folder to short one * Adding comments for ssmt tests * add idmtools tags to tests, update inputdata and era5 climate tests * fix import issue with test_ssmt_python.py * remove unused folder which cost some test failed * SSMT: fix platform unit tests * SSMT: fix task issue * SSMT: fix test_comps_plugin.py issue * Fix all failed tests in idmtoold_platform_comps * Change run test to be able to switch between idmtools and dtktools Co-authored-by: mfisher-idmod <37818827+mfisher-idmod@users.noreply.github.com> Co-authored-by: Sharon Chen <38014764+shchen-idmod@users.noreply.github.com>
- Loading branch information