Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration tests setup, updated docs #51

Merged
merged 5 commits into from
Nov 1, 2018
Merged

Conversation

apaleyes
Copy link
Collaborator

Issue #, if available: #32

Description of changes:

  • Replaced RF model with scikit-learn
  • Fixed requirements docs
  • Setup integration tests run during the build
  • Removed examples from coverage reports. It is not trivial to track, and it is questionable whether we should be tracking coverage of examples as strict as the core library
  • As a side effect, updated docs to include recent changes

Testing: made sure tests, integration tests and docs all run locally

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@marpulli
Copy link
Contributor

Seems like enum.auto was introduced in 3.6. We can either not support 3.5 or remove the use of auto.

Copy link
Contributor

@marpulli marpulli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good apart from tests failing due to logs being too long and using enum.auto

.travis.yml Outdated
@@ -27,13 +27,15 @@ install:
- pip install --upgrade pip
- pip install -r requirements/requirements.txt
- pip install -r requirements/test_requirements.txt
- pip install -r requirements/integration_test_requirements.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To stop the linux logs reaching the size limit, maybe making the pip install less verbose would work?

Suggested change
- pip install -r requirements/integration_test_requirements.txt
- pip install -r requirements/integration_test_requirements.txt -q

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good idea, thanks. Will certainly give it a try.

pip install git+https://github.com/automl/random_forest_run.git
pip install scikit-learn

Refer to http://scikit-learn.org/stable/install.html for further information.
""")


class RandomForest(IModel):

def __init__(self, X_init: np.ndarray, Y_init: np.ndarray, num_trees: int = 30,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you feel about allowing the user to pass in kwargs for the sklearn model? We don't want to expose all options in the sklearn random forest constructor but we are limiting what a user can do here.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I feel negatively, but I can be persuaded...

Anyways, the sole point of existence of these models, from my point of view, was to give users something simple to play with out of the box. They will be limiting no doubt, but then they are not intended for a real modelling.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if they can be made ready for "real" modelling with such a small change, shouldn't we do it? I think it would be quite frustrating if I had to re-implement this class just so I can pass in one more parameter to the sklearn random forest constructor.

An alternative might be to have a "SciKitLearnRandomForestWrapper" and simply have a function that creates a random forest with default parameters, passes it in to the wrapper then returns this wrapper to the user. Then if a user wants to create a custom random forest model they can reuse the wrapper.

@apaleyes
Copy link
Collaborator Author

@marpulli I was going to get rid of auto(), feels wrong to rule out 3.5 just because of it

@codecov-io
Copy link

codecov-io commented Nov 1, 2018

Codecov Report

Merging #51 into develop will increase coverage by 8.81%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop      #51      +/-   ##
===========================================
+ Coverage     81.2%   90.01%   +8.81%     
===========================================
  Files           62       58       -4     
  Lines         1532     1382     -150     
  Branches       154      140      -14     
===========================================
  Hits          1244     1244              
+ Misses         253      103     -150     
  Partials        35       35
Impacted Files Coverage Δ
emukit/examples/__init__.py

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4a43c47...5a922bd. Read the comment docs.

@apaleyes
Copy link
Collaborator Author

apaleyes commented Nov 1, 2018

"will increase coverage by 8.81%." yay!

@apaleyes apaleyes merged commit 39b17d7 into develop Nov 1, 2018
@apaleyes apaleyes deleted the apaleyes/opt-deps branch November 1, 2018 17:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants