Fuzzy string search for Indian languages
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
docs
libindic/inexactsearch
.gitignore
.testr.conf
.travis.yml
Makefile
README.md
inexactsearchtest.py
requirements.txt
setup.cfg
setup.py
test-requirements.txt

README.md

LibIndic Inexact Search

Build Status Coverage Status

LibIndic's inexactsearch module is a Fuzzy string search. This application illustrates the combined use of Edit distance and Indic Soundex algorithm.

By mixing both written like(edit distance) and sounds like(soundex), we achieve an efficient aproximate string searching. This application is capable of cross language string search too. That means, you can search Hindi words in Malayalam text. If there is any Malayalam word, which is approximate transliteration of hindi word, or sounds alike the hindi words, it will be returned as an approximate match. The "written like" algorithm used here is a bigram average algorithm. The ratio of common bigrams in two strings and average number of bigrams will give a factor which is greater than zero and less than 1. Similarly the soundex algorithm also gives a weight. By selecting words which has comparison weight more than the threshold weight(which 0.6), we get the search results.

Installation

  1. Clone the repository git clone https://github.com/libindic/inexactsearch.git
  2. Change to the cloned directory cd inexactsearch
  3. Run setup.py to create installable source python setup.py sdist
  4. Install using pip pip install dist/libindic-inexactsearch*.tar.gz

Usage

>>> from libindic.inexactsearch import InexactSearch
>>> instance = InexactSearch()
>>> result = instance.search(u"कालडी केरल के एर्नाकुलम जिले में पेरियार नदी के पूर्व में स्थित एक गाँव है", u"കാലടി")
>>> matches = [word for word, similarity in result.items() if similarity > 0.8]
>>> for match in matches:
...     print(match)
...
कालडी

For more details read the docs