Tags: microsoft/MLOS
Tags
Improve mlos-viz for multiple repeats of a config and add tests (#633) - Mark `mlos_viz` as `typed` for `mypy` - Bump version - Mock calls to matplotlib/dabl for testing - Add plotting of top-N configs - Improve plots for handling repeat config trials via variance error bars --------- Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
Refactor `mlos_bench.storage` and add `TunableConfigTrialGroup` prope… …rty for `TrialData` and `ExperimentData` (#648) Useful for grouping trials by the config they used. In use by upcoming #633 for generating graphs with variance error bars for repeated configs. Also refactors a number of other things: - Standardize on `experiment_id` instead of `exp_id` (but not the db schema for now) - Standardize on `tunable_config_id` instead of `config_id` in the API since we also call it `tunable_config` for the object fetching property to distinguish from the `config` dict used internally. (but not the db schema for now) - Rework the idea of `TunableConfigTrialGroup` as an object inaddition to an ID (further methods can be added later to move back and forth between types when doing interactive analysis). - Rework the idea of a `TunableConfig` as an object for fetching tunable value assignments (similar justification - easier grouping in the future by fetching trial across experiments based on config - eventually could be used to house the experiment merge logic). - Rename `results` APIs to `results_df` (similar for others that return `pandas.DataFrame`) to match the `results_dict` that return `dict` - Refactor test fixtures to match other styles and for future use (moved to #644). - Expand tests NOTE: - We cut a new version with this commit since there are potentially breaking API changes (e.g., `results` -> to `results_df` and `exp_id` -> `experiment_id`). Currently builds off of #644 and splits work out of #633 --------- Co-authored-by: Sergiy Matusevych <sergiym@microsoft.com>
PyPi publish makefile rules fixup (#640) Prior Makefile rule consolidation tried to group dependencies for the pattern rule into separate lines for readability, but apparently this isn't allowed. This minor changes separates those out into a variable we can reference instead. Tested on a clean build tree locally.
Support for special tunable values outside of the range (#617) Enables special values outside of the range (e.g., `-1` with a range of `[0, 100]`). To do we make use of "conditionals" in ConfigSpace to constrain the space. This has a number of implementation implications, addressed below: * [x] Add support for special values to the `Tunable` class * [x] Add unit tests for assigning special values outside of the range to the `Tunable` objects * [x] Add special values outside of the range to the unit tests for `ConfigSpace` conversion * [x] Implement proper `TunableGroups` to `ConfigSpace` conversion for tunables with special values * [x] Update `mlos_core` optimizers to support conditionals and special values in `ConfigSpace` * [x] Add more unit tests to check the conversion * [x] Make LlamaTune adapter support conditionals in `ConfigSpace` --------- Co-authored-by: Brian Kroth <bpkroth@users.noreply.github.com>
PreviousNext