Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Summary: 1. add call to binarization to complete preprocessing pipeline 2. add ability to specify task to select the dictionary, and add a bert task 3. Get rid of function calls that are no longer needed after moving functions from fairseq here Reviewed By: jingfeidu Differential Revision: D13977842 fbshipit-source-id: ec9bbb4e98e62e12c20ba68bb52b8bcc94aee91d
- Loading branch information