-
Notifications
You must be signed in to change notification settings - Fork 394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow TextStats length distribution to be token-based and refactor for testability #464
Merged
Merged
Changes from 24 commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
1c2cbc2
Refactored and added more incremental tests
Jauntbox 83881c0
Updated test
Jauntbox ca2e122
Added tests and fixed a small bug
Jauntbox c190d97
More refactoring and updating tests
Jauntbox 75b0770
More test refactoring
Jauntbox 305c57e
More refactoring
Jauntbox ff53dc4
Small cleanups
Jauntbox 13b6076
Merge branch 'master' of github.com:salesforce/TransmogrifAI into km/…
Jauntbox fc6cd07
Addressing comments
Jauntbox 1434128
Added text length distribution to the TextStats calculated in RFF
Jauntbox e117fbd
Made offending methods private
Jauntbox 87878e0
Merge branch 'master' of github.com:salesforce/TransmogrifAI into km/…
Jauntbox fe36ec5
Comments
Jauntbox cd31e32
Spelling
Jauntbox a322363
Added toggle for tokenization in length distribution
Jauntbox 32ad893
Added toggle to turn tokenization on/off for length distribution coun…
Jauntbox ab9d2a7
Reverted changes to RFF for now and added logging to help with visibi…
Jauntbox 4cb5c88
Updated tests to check both tokenized and non-tokenized text feature …
Jauntbox e463685
Better logging
Jauntbox e44ca68
Revert unintentional RFF changes
Jauntbox ce5663e
Removed unused method
Jauntbox b866bb3
Removed SVC models from the default models to try in BinaryClassifica…
Jauntbox e72af36
Added new params to vectorizer shortcuts
Jauntbox 307a014
scalastyle issue
Jauntbox 49127d9
Replaced boolean param with enum
Jauntbox 95bc3e7
Added enum to json4s serialization list
Jauntbox aad13ba
Actually add the enum file
Jauntbox d78868d
Merge branch 'master' of github.com:salesforce/TransmogrifAI into km/…
Jauntbox File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we make this an enum rather than a boolean? then we have room to expend in the future