4 branches 62 tags

Code

This branch is 5 commits ahead, 53 commits behind explosion:master.

244 commits

Failed to load latest commit information.

README.md

⚠ MOVED

This repository was moved to https://github.com/explosion/tokenizations, and now maintained by Explosion, spaCy's authors.

Robust and Fast tokenizations alignment library for Rust and Python

Demo: demo
Rust document: docs.rs
Blog post: How to calculate the alignment between BERT and spaCy tokens effectively and robustly

About

Robust and Fast tokenizations alignment library for Rust and Python https://tamuhey.github.io/tokenizations/

Report repository

Releases

Packages

No packages published

Languages

Rust 71.1%
Python 20.7%
Dockerfile 7.4%
Makefile 0.8%