Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added onnx export support for WordTokenizingTransformer and NgramExtractingTransformer #4451

Merged
merged 3 commits into from Nov 13, 2019

Conversation

@harishsk
Copy link
Member

harishsk commented Nov 7, 2019

PR contents:

  • Onnx export support for for WordTokenizingTransformer and NgramExtractingTransformer
  • Related bug fixes in ValueToKeyMappingTransformer
  • Bug fixes for string handling in OnnxUtils.cs
  • Related unit tests
Harish S. Kulkarni added 3 commits Oct 25, 2019
A large portion of this commit is upgrading the baseline test files. The rest of the fixes deal with build breaks resulting from the upgrade of ORT version.
@harishsk harishsk requested a review from dotnet/mlnet-core as a code owner Nov 7, 2019
@harishsk harishsk requested review from ganik, eerhardt and codemzs Nov 7, 2019
@codecov

This comment has been minimized.

Copy link

codecov bot commented Nov 7, 2019

Codecov Report

Merging #4451 into master will decrease coverage by 15.3%.
The diff coverage is 97.52%.

@@             Coverage Diff             @@
##           master    #4451       +/-   ##
===========================================
- Coverage   90.11%   74.81%   -15.31%     
===========================================
  Files         200      906      +706     
  Lines       36848   159549   +122701     
  Branches     1910    17177    +15267     
===========================================
+ Hits        33206   119365    +86159     
- Misses       3337    35390    +32053     
- Partials      305     4794     +4489
Flag Coverage Δ
#Debug 74.81% <97.52%> (-15.31%) ⬇️
#production 70.19% <96.47%> (?)
#test 90.14% <99.11%> (+0.02%) ⬆️
Impacted Files Coverage Δ
src/Microsoft.ML.OnnxTransformer/OnnxUtils.cs 90.04% <100%> (ø)
src/Microsoft.ML.Data/Transforms/KeyToVector.cs 83.74% <100%> (ø)
src/Microsoft.ML.Transforms/Text/WordTokenizing.cs 80.78% <100%> (ø)
...ML.Data/Transforms/ValueToKeyMappingTransformer.cs 78.75% <94.44%> (ø)
src/Microsoft.ML.Transforms/Text/NgramTransform.cs 91.47% <95.74%> (ø)
test/Microsoft.ML.Tests/OnnxConversionTest.cs 94.85% <99.11%> (+0.67%) ⬆️
src/Microsoft.ML.Data/Model/Pfa/ModelUtils.cs 0% <0%> (ø)
...ML.Data/Transforms/ColumnConcatenatingEstimator.cs 83.58% <0%> (ø)
src/Microsoft.ML.Data/Model/Pfa/PfaUtils.cs 0% <0%> (ø)
...t.ML.Data/DataLoadSave/CompositeLoaderEstimator.cs 80% <0%> (ø)
... and 702 more
@ganik
ganik approved these changes Nov 13, 2019
Copy link
Member

ganik left a comment

:shipit:

@harishsk harishsk merged commit 693250b into dotnet:master Nov 13, 2019
19 checks passed
19 checks passed
MachineLearning-CI Build #20191106.13 had test failures
Details
MachineLearning-CI (Centos_x64_NetCoreApp30 Debug_Build) Centos_x64_NetCoreApp30 Debug_Build succeeded
Details
MachineLearning-CI (Centos_x64_NetCoreApp30 Release_Build) Centos_x64_NetCoreApp30 Release_Build succeeded
Details
MachineLearning-CI (MacOS_x64_NetCoreApp21 Debug_Build) MacOS_x64_NetCoreApp21 Debug_Build succeeded
Details
MachineLearning-CI (MacOS_x64_NetCoreApp21 Release_Build) MacOS_x64_NetCoreApp21 Release_Build succeeded
Details
MachineLearning-CI (Ubuntu_x64_NetCoreApp21 Debug_Build) Ubuntu_x64_NetCoreApp21 Debug_Build succeeded
Details
MachineLearning-CI (Ubuntu_x64_NetCoreApp21 Release_Build) Ubuntu_x64_NetCoreApp21 Release_Build succeeded
Details
MachineLearning-CI (Windows_x64_NetCoreApp21 Debug_Build) Windows_x64_NetCoreApp21 Debug_Build succeeded
Details
MachineLearning-CI (Windows_x64_NetCoreApp21 Release_Build) Windows_x64_NetCoreApp21 Release_Build succeeded
Details
MachineLearning-CI (Windows_x64_NetCoreApp30 Debug_Build) Windows_x64_NetCoreApp30 Debug_Build succeeded
Details
MachineLearning-CI (Windows_x64_NetCoreApp30 Release_Build) Windows_x64_NetCoreApp30 Release_Build succeeded
Details
MachineLearning-CI (Windows_x64_NetFx461 Debug_Build) Windows_x64_NetFx461 Debug_Build succeeded
Details
MachineLearning-CI (Windows_x64_NetFx461 Release_Build) Windows_x64_NetFx461 Release_Build succeeded
Details
MachineLearning-CI (Windows_x86_NetCoreApp21 Debug_Build) Windows_x86_NetCoreApp21 Debug_Build succeeded
Details
MachineLearning-CI (Windows_x86_NetCoreApp21 Release_Build) Windows_x86_NetCoreApp21 Release_Build succeeded
Details
MachineLearning-CodeCoverage Build #20191106.13 succeeded
Details
MachineLearning-CodeCoverage (Windows_x64 Build_Debug) Windows_x64 Build_Debug succeeded
Details
WIP Ready for review
Details
license/cla All CLA requirements met.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.