-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add config filter parameter to simulate_factors() #14
Merged
Commits on May 1, 2017
-
New only_factor parameter for simulate_factors()
Add a new optional `only_factor` parameter for the simulate_factors() method. The only_factor parameter is specified, only a single simulation will be run. The simulation that runs is for the factor at the index specified by the value of the only_factor argument. For example, if the snowbird command-line arguments look like this: meta.cmd_line: sb -f host.queue_depth 4,16 -f fm.fim.xfer_rate_MBps 400,800 -f fm.ase.t_wait_rd 4,5,6 -s duration 10 ms --only-factor 4 Then snowbird will pass in only_factor=4 as the argument to desmod and the meta.sim.index and meta.sim.workspace will be set to 4 and workspace/4 respectively. Additionally, meta.sim.special will look like this: meta.sim.special: - [host.queue_depth, 4] - [fm.fim.xfer_rate_MBps, 800] - [fm.ase.t_wait_rd, 5] That is, only a single simulation, for the factor indicated by meta.sim.special, was run. One use for this new feature is orchestrating the execution of multiple, per-factor, simulations on separate machines (e.g., in the cloud) with a single command (with multiple factors).
Configuration menu - View commit details
-
Copy full SHA for 3142c63 - Browse repository at this point
Copy the full SHA 3142c63View commit details -
Optionally sync workspaces to s3
Some new config options have been added: - sim.workspace.s3_sync: When true, syncs any workspaces to s3 - sim.workspace.s3_bucket: The s3 bucket to sync to - sim.workspace.s3_sync_prefix: A string to act as the root path in the s3 bucket. By default, this is set the user's username. Here's an example of running snowbird with these new options: $ sb -s sim.workspace ~/locker/test -s sim.workspace.s3_sync_prefix bgmerrell -n test-rd-rnd -s sim.duration 10ms -s sim.workspace.s3_sync True -f host.queue_depth 4,16 This would result in the following being synced to the default bucket in s3: /bgmerrell/test/0/config.yaml /bgmerrell/test/0/result.yaml /bgmerrell/test/0/sim.log /bgmerrell/test/1/config.yaml /bgmerrell/test/1/result.yaml /bgmerrell/test/1/sim.log The method to do the sync work is called when the _Workspace context manager is exiting. The uploading to s3 is done using the concurrent.features package. This package isn't available in the Python 2 library, so the requirements.txt file has been updated to install the package for Python 2 only.
Configuration menu - View commit details
-
Copy full SHA for ab8ab48 - Browse repository at this point
Copy the full SHA ab8ab48View commit details -
Configuration menu - View commit details
-
Copy full SHA for 190ff12 - Browse repository at this point
Copy the full SHA 190ff12View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3383cf5 - Browse repository at this point
Copy the full SHA 3383cf5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 006567d - Browse repository at this point
Copy the full SHA 006567dView commit details
Commits on May 3, 2017
-
* only_factor -> config_filter * use setdefault() for some workspace sync config keys * default s3 sync prefix is now "" * don't importa boto3 or concurrent.futures until needed. * sim.workspace.s3_sync -> sim.sync.s3.enable (and similar for other s3 sync-related keys)
Configuration menu - View commit details
-
Copy full SHA for 49242a8 - Browse repository at this point
Copy the full SHA 49242a8View commit details
Commits on May 5, 2017
-
The sync code will not live in desmod. We've decided to take a different approach.
Configuration menu - View commit details
-
Copy full SHA for a1a33b8 - Browse repository at this point
Copy the full SHA a1a33b8View commit details -
Configuration menu - View commit details
-
Copy full SHA for aa35322 - Browse repository at this point
Copy the full SHA aa35322View commit details
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.