Distil-Lang-Detect

Distil-lang-detect is text language detection module based on sequence classification technique DistilBERT by 🤗 Huggingface Transformers.

Getting Started

Distil-Lang-Detect can be easily fired-up. Just need to the following.

Requirements

• python 3.5
• torch >= 1.2.0
• transformers >= 2.2.2

Installation

git clone https://github.com/yash1994/distil-lang-detect.git
cd dframcy
python setup.py install

Usage

from distillangdetect.detector import Detector
dct = Detector(device="cpu")
det = dct.detect("I love retro computing.")
print(det)

'English'

Todos

• Extensive testing.
• Add training and evaluation scripts.
• Output format options.
• Batch Processing.
• Bechmarking on different datasets.