Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix MatrixFactorization trainer's warning #5071

Conversation

LittleLittleCloud
Copy link
Contributor

@LittleLittleCloud LittleLittleCloud commented Apr 29, 2020

Problem to fix

MatrixFactorization's default options will throw a warning msg Warning: insufficient blocks may slow down the trainingprocess (4*nr_threads^2+1 blocks is suggested) when running on PC with more than 10 threads.

Cause

The warning msg is from libmf, which will check nr_bin and nr_thread, if nr_bin <= 2* nr_thread, it will show that warning.

MatrixFactorization uses max(20, 2*nr_thread) to initialize nr_bin, when nr_thread < 10, it's fine because nr_bin will be 20, which is greater than 2*nr_thread, but if nr_thread is >= 10, nr_bin's value will be 2*nr_thread, which is less than 2*nr_thread + 1 and trigger warning.

Fix

initialize nr_bin with 2 * nr_threads + 1.

Related issue

#5067

@LittleLittleCloud LittleLittleCloud requested a review from a team as a code owner April 29, 2020 18:13
@codecov
Copy link

codecov bot commented May 4, 2020

Codecov Report

Merging #5071 into master will decrease coverage by 0.18%.
The diff coverage is 54.95%.

@@            Coverage Diff             @@
##           master    #5071      +/-   ##
==========================================
- Coverage   75.83%   75.65%   -0.19%     
==========================================
  Files         951      993      +42     
  Lines      172613   178157    +5544     
  Branches    18632    19124     +492     
==========================================
+ Hits       130904   134786    +3882     
- Misses      36529    38145    +1616     
- Partials     5180     5226      +46     
Flag Coverage Δ
#Debug 75.65% <54.95%> (-0.19%) ⬇️
#production 71.62% <54.95%> (+0.17%) ⬆️
#test 88.67% <ø> (-1.83%) ⬇️
Impacted Files Coverage Δ
....AutoML/API/RunDetails/CrossValidationRunDetail.cs 100.00% <ø> (ø)
...rc/Microsoft.ML.AutoML/API/RunDetails/RunDetail.cs 95.45% <ø> (ø)
....AutoML/ColumnInference/ColumnGroupingInference.cs 85.00% <ø> (ø)
...t.ML.AutoML/ColumnInference/ColumnTypeInference.cs 85.46% <ø> (+1.76%) ⬆️
...rosoft.ML.AutoML/ColumnInference/TextFileSample.cs 62.25% <0.00%> (ø)
...icrosoft.ML.AutoML/Experiment/SuggestedPipeline.cs 88.65% <ø> (ø)
....ML.AutoML/PipelineSuggesters/PipelineSuggester.cs 83.19% <ø> (ø)
src/Microsoft.ML.AutoML/Sweepers/Parameters.cs 84.32% <ø> (ø)
src/Microsoft.ML.AutoML/Sweepers/SmacSweeper.cs 93.47% <ø> (ø)
src/Microsoft.ML.AutoML/Utils/BestResultUtil.cs 57.53% <0.00%> (-1.63%) ⬇️
... and 363 more

@LittleLittleCloud LittleLittleCloud merged commit f560585 into dotnet:master May 6, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants