New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update SplitMode implementation #243

Closed

mh-northlander opened this issue Feb 15, 2024 · 1 comment · Fixed by #245

Labels

python

Collaborator

mh-northlander commented Feb 15, 2024

The issue around enum on PyO3 side seems solved: PyO3/pyo3#834.

sudachi.rs/python/src/tokenizer.rs

Lines 38 to 39 in d09dd00

    
           // This implementation is a workaround. Waiting for the pyo3 enum feature. 
        
           // ref: [PyO3 issue #834](https://github.com/PyO3/pyo3/issues/834).

Check if we can use it for the PySplitMode.

mh-northlander added the python label

Collaborator

eiennohito commented Feb 15, 2024

Probably should also accept string literals, "A", "B", "C" as well.

eiennohito mentioned this issue

Allow string literals as segmentation modes #245

Merged

mh-northlander closed this as completed in #245

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment