Stemmer for Marathi

[the-ethan-hunt]

I would be adding the stemmer for Marathi language with the testing file also.

[kylepjohnson]

Sure, go for it, but could you explain a little bit more about how you will do this.

[the-ethan-hunt]

@kylepjohnson , if we compare English and Marathi:

I run : मी दवडतो
I ran: मी दवडला

Here the verb is 'दवड' while the suffixes are 'तो' and 'ला'.
A suffix stripping algorithm is ineffective in English(the example of run and ran as given here); however it very effective in Marathi as suffixes are added to convert a verb form into past, present or the future. Once the suffixes are stripped, we get the verb.
Would love to hear about this approach from you. 😄

[kylepjohnson]

Hi any work done on this yet? Rule-based stemming is challenging but how it has been done in the past.

For inspiration look at the Stemmer() for Latin, which @lukehollis did some years back.

And you should look at stemmers for other, similar Indian languages. For example, I read that Lucene uses the algorithm described in this paper: http://computing.open.ac.uk/Sites/EACLSouthAsia/Papers/p6-Ramanathan.pdf

There's some been some work on modern Marathi, though the burden would be upon you to prove to us that these would either (a) work on older Marathi or (b) could be adapted by you for pre-modern Marathi.

• http://iosrjournals.org/iosr-jce/papers/Conf.17003/Volume-3/10.%2051-54.pdf
• http://www.enggjournals.com/ijcse/doc/IJCSE10-02-08-116.pdf
• https://www.academia.edu/535847/Discovering_suffixes_A_Case_Study_for_Marathi

[the-ethan-hunt]

@kylepjohnson , yes a PR is pending for review. Please see PR #719