Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Wrapper to use hugging face algorithms for building vocabulary #3316

Merged
merged 2 commits into from
Apr 11, 2023
Merged

Wrapper to use hugging face algorithms for building vocabulary #3316

merged 2 commits into from
Apr 11, 2023

Conversation

arunppsg
Copy link
Contributor

Pull Request Template

Description

A rework of #3271

Type of change

Please check the option that is related to your PR.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • In this case, we recommend to discuss your modification on GitHub issues before creating the PR
  • Documentations (modification for documents)

Checklist

  • My code follows the style guidelines of this project
    • Run yapf -i <modified file> and check no errors (yapf version must be 0.32.0)
    • Run mypy -p deepchem and check no errors
    • Run flake8 <modified file> --count and check no errors
    • Run python -m doctest <modified file> and check no errors
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • New unit tests pass locally with my changes
  • I have checked my code and corrected any misspellings

@arunppsg arunppsg marked this pull request as draft March 29, 2023 11:44
@arunppsg arunppsg force-pushed the hf-vocab-3 branch 2 times, most recently from 56222b2 to 365e39b Compare April 10, 2023 11:39
@arunppsg arunppsg marked this pull request as ready for review April 10, 2023 11:41
rbharath
rbharath approved these changes Apr 11, 2023
View reviewed changes
Copy link
Member

@rbharath rbharath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, feel free to merge in once CI is clear

arunppsg and others added 2 commits April 11, 2023 12:25
@arunppsg
added huggingface vocabulary builder
2021d5c 
added test for huggingface vocabulary builder
added hf_vocab to init
added docs
added tokenizers to dependency
b27c017 
added tokenizer to soft requirements
@arunppsg arunppsg merged commit 4e2839c into deepchem:master Apr 11, 2023
24 of 36 checks passed
@arunppsg arunppsg deleted the hf-vocab-3 branch April 11, 2023 09:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rbharath rbharath rbharath approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@arunppsg @rbharath