Skip to content

RNogales94/cld3

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

63 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

CLD3 Python bindings

These are Python bindings for cld3, a language classifying library used in Google Chrome.

Included are Python bindings (via Cython). Building the extension does not require the chromium repository, instead only these libraries need to be installed:

  • Cython
  • msgpack
  • Protobuf (with headers and protoc)

To install the extension, just run pip install on the repository URL, or use pip install cld3.

Usage

Here's some examples:

>>> cld3.get_language("This is a test")
LanguagePrediction(language='en', probability=0.9999980926513672, is_reliable=True, proportion=1.0)

>>> cld3.get_frequent_languages("This piece of text is in English. Този текст е на Български.", 5)
[LanguagePrediction(language='bg', probability=0.9173890948295593, is_reliable=True, proportion=0.5853658318519592), LanguagePrediction(language='en', probability=0.9999790191650391, is_reliable=True, proportion=0.4146341383457184)]

In short:

get_language returns the most likely language as the named tuple LanguagePrediction. Proportion is always 1.0 when called in this way.

get_frequent_languages will return the top number of guesses, up to a maximum specified (in the example, 5). The maximum is mandatory. Proportion will be set to the proportion of bytes found to be the target language in the list.

In the normal cld3 library, "und" may be returned as a language for unknown languages (with no other stats given). This library filters that result out as extraneous; if the language couldn't be detected, nothing will be returned. This also means, as a consequence, get_frequent_languages may return fewer results than what you asked for, or none at all.

About

Python bindings for cld3

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 99.8%
  • Python 0.2%