feat: add matrix type parameter and improve auto logic #1052
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
imatiach-msft
commented
May 17, 2021
imatiach-msft
commented
- add matrix type parameter to specify whether matrix is sparse, dense, or auto inferred
- allow user to override default auto logic to specify whether sparse or dense matrix must be constructed in lightgbm native code
- improve autologic to use first 10 sampled rows instead of just the head row when trying to determine whether constructed matrix should be sparse or dense
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
imatiach-msft
force-pushed
the
ilmat/matrix-type
branch
2 times, most recently
from
eeab0b2
to
5beeffe
Compare
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report
@@ Coverage Diff @@
## master #1052 +/- ##
==========================================
+ Coverage 84.93% 84.96% +0.02%
==========================================
Files 203 203
Lines 9677 9689 +12
Branches 588 558 -30
==========================================
+ Hits 8219 8232 +13
+ Misses 1458 1457 -1
Continue to review full report at Codecov.
|
imatiach-msft
force-pushed
the
ilmat/matrix-type
branch
from
5beeffe
to
8a115f3
Compare
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
imatiach-msft
force-pushed
the
ilmat/matrix-type
branch
from
8a115f3
to
8cd8007
Compare
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
imatiach-msft
commented
May 17, 2021
| @@ -320,12 +320,15 @@ class VerifyLightGBMClassifier extends Benchmarks with EstimatorFuzzing[LightGBM | |||
| test("Verify LightGBM Classifier with dart mode parameters") { | |||
| // Assert the dart parameters work without failing and setting them to tuned values improves performance | |||
| val Array(train, test) = pimaDF.randomSplit(Array(0.8, 0.2), seed) | |||
| val scoredDF1 = baseModel.setBoostingType("dart").fit(train).transform(test) | |||
| val scoredDF1 = baseModel.setBoostingType("dart"). | |||
There was a problem hiding this comment.
note: this is just fixing a flaky test in the build, unrelated to this PR
mhamilton723
requested changes
May 17, 2021
src/main/scala/com/microsoft/ml/spark/lightgbm/TrainUtils.scala
Outdated
Show resolved
Hide resolved
imatiach-msft
force-pushed
the
ilmat/matrix-type
branch
from
8cd8007
to
912b6e8
Compare
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
imatiach-msft
changed the title
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
mhamilton723
approved these changes
May 17, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.