-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding support for unigram sentencepiece model #51
Conversation
|
Hi @Ayushk4, |
Travis CI fails on Julia 1.0, due to new DataDeps as dependency - requires updating Project.toml. |
@oxinabox Should we drop support for 0.7 tests now in Appveyor? |
Since DataDeps have been newly added, Travis and Appveyor also need to be updated to allow DataDeps downloads |
Yes |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please look into setting DATADEPS variable for Travis and Appveyor CI. Also remove 0.7 Julia tests for appveyor.
Co-authored-by: Ayush Kaushal <ayushk4@gmail.com>
Co-authored-by: Ayush Kaushal <ayushk4@gmail.com>
Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
Thank you. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Cool just a few last things this this should be good to go.
@Ayushk4 can merge once happy with it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me.
As discussed in #44
I am adding support for the sentencepiece unigram model with
DataDeps
to download pre-trained vocab file with their log-likelihoodfor more detail refer to the blog post here
Roadmap