Skip to content

automl/ParameterImportance

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
March 28, 2018 15:30
April 1, 2020 14:07
January 12, 2018 19:04
March 26, 2021 16:47
April 1, 2020 14:07
May 29, 2018 13:46
November 30, 2016 11:14
January 19, 2018 15:57
April 1, 2020 14:07
June 29, 2020 15:49
January 19, 2018 16:41

PyImp

PyImp is now on pypi. To install it simply run pip install pyimp If you want to run fANOVA with PyImp you will have to manually install it via pip install git+http://github.com/automl/fanova.git@master

PyImp is an easy to use tool that helps developers to identify the most important parameters of their algorithms. Given the data of a configuration run with SMAC3, PyImp allows one to use Forward Selection, Efficient Ablation and Influence Models to determine which Parameters have the most influence over the algorithms behaviour.

PyImp can be used with argcomplete. To enable autocompletion of PyImp arguments, add the following line to your .bashrc or .profile: eval "$(register-python-argcomplete pyimp)"

The documentation can be found here.

Example results of the package look as follows:

Forward Selection

An example call of forward-selection:

cd examples/branin
pimp --scenario_file smac3-out/scenario.txt --history smac3-out/runhistory.json --modus forward-selection

Results in an image such as:

Surrogate-ablation

An example call of surrogate-ablation:

cd examples/branin
pimp --scenario_file smac3-out/scenario.txt --history smac3-out/runhistory.json --trajectory smac3-out/traj.json --modus ablation`

Results in two plots:

Reading in multiple runhistories

To read in multiple runhistories, check out the spear-qcp example:

cd examples/spear_qcp
pimp --scenario_file scenario.txt --history './*/runhistory.json' --modus forward-selection