New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation scenario tests #2503

Merged
merged 12 commits into from Feb 13, 2019

Conversation

Projects
None yet
3 participants
@rogancarr
Copy link
Contributor

rogancarr commented Feb 11, 2019

This PR adds a test for training with a validation set.

Fixes #2499

@codecov

This comment has been minimized.

Copy link

codecov bot commented Feb 11, 2019

Codecov Report

Merging #2503 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #2503      +/-   ##
==========================================
+ Coverage   71.25%   71.25%   +<.01%     
==========================================
  Files         798      797       -1     
  Lines      141252   141270      +18     
  Branches    16112    16112              
==========================================
+ Hits       100643   100658      +15     
- Misses      36145    36151       +6     
+ Partials     4464     4461       -3
Flag Coverage Δ
#Debug 71.25% <100%> (ø) ⬆️
#production 67.57% <ø> (-0.01%) ⬇️
#test 85.36% <100%> (ø) ⬆️
@Ivanidzo4ka
Copy link
Member

Ivanidzo4ka left a comment

:shipit:

@glebuk glebuk self-requested a review Feb 12, 2019

var preprocessedValidData = preprocessor.Transform(validData);

// Train the model with a validation set
var trainedModel = mlContext.Regression.Trainers.FastTree(numTrees: 2)

This comment has been minimized.

@glebuk

glebuk Feb 12, 2019

Contributor

trainedModel [](start = 16, length = 12)

add early stopping criteria in the options so that the validation set is actually useful.
Such as https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.trainers.fasttree.boostedtreeargs.earlystoppingrule?view=ml-dotnet #Closed

// Combine the model
var model = preprocessor.Append(trainedModel);

// Score the data sets

This comment has been minimized.

@glebuk

glebuk Feb 12, 2019

Contributor

sets [](start = 30, length = 4)

periods at the end of comments throughout. #Closed

rogancarr added some commits Feb 13, 2019

@glebuk

glebuk approved these changes Feb 13, 2019

Copy link
Contributor

glebuk left a comment

:shipit:

@rogancarr rogancarr merged commit e818e35 into dotnet:master Feb 13, 2019

3 checks passed

MachineLearning-CI #20190212.39 succeeded
Details
MachineLearning-CodeCoverage #20190212.39 succeeded
Details
license/cla All CLA requirements met.
Details

@rogancarr rogancarr deleted the rogancarr:2499_validation_scenario_tests branch Feb 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment