LibIndic's stemmer module may be used to extract stems of the words in a sentence. It is implemented in a rule-based model and follows iterative suffix stripping to handle multiple levels of inflection. Right now, it supports Malayalam language only.
- Clone the repository
git clone https://github.com/libindic/indicstemmer.git - Change to the cloned directory
cd indicstemmer - Run setup.py to create installable source
python setup.py sdist - Install using pip
pip install dist/libindic-stemmer*.tar.gz
Note: Prefer using virtualenv for installation as the library is in experimental stage
Input: String <str> containing words word1 word2 word3 ...
Output: Dict <dict> of the format
{
'word1': {
'stem': 'stem1',
'inflection': ['tag1', 'tag2', ...]
},
'word2': {
'stem': 'stem2',
'inflection': ['tag1', 'tag2', ...]
},
.
.
.
}
>>> from libindic.stemmer import Stemmer
>>> stemmer = Stemmer()
>>> result = stemmer.stem(language='malayalam', text=രാമന്റെ വീട്ടിലേക്ക്')
>>> for word, output in result.items():
... print word, " : ", output['stem'], " : ", output['inflection']
രാമന്റെ : രാമൻ : ['SAMB1']
വീട്ടിലേക്ക് : വീട് : ['MISC1', 'ADH1', 'UDH1']
For more details read the docs