Plural Stemmer for English

[jgschis]

The snowball and porter stemmers are too aggressive for ecommerce. For example, the word dressing gets stemmed to dress. But dressing and dress are two different concepts and shouldn't be conflated.

I think we need a stemmer that just reduces a plural to its singular form. The OpenSearch project recently added a stemmer that does this:
https://github.com/opensearch-project/OpenSearch/blob/main/modules/analysis-common/src/main/java/org/opensearch/analysis/common/EnglishPluralStemFilter.java

If no one else wants to do this, I will add this to Bleve...

[abhinavdangeti]

@jgschis Thanks for bringing this up.

We'd welcome a contribution from you. If not, we will look into this eventually.

[jgschis]

Yes I am doing it now...

[jgschis]

I have ported the code to go. This is the first time i have made a go program, so no sure if I've done this right

https://github.com/jgschis/pluralstem/blob/master/english/english.go

[jgschis]

HI,
I am now adding an "entry point" to the plural stemmer in Bleve.

Should I

• Add my code to https://github.com/blevesearch/bleve/tree/master/analysis/token
• Add my code to https://github.com/blevesearch/bleve/tree/master/analysis/lang/en

[jgschis]

Feel free to add this to the bleve project
https://github.com/jgschis/pluralstem

once done, i will make a pull request to add the entry point to this library to Bleve.

[jgschis]

Hi,

@abhinavdangeti
What has to happen for my code to become part of bleve? I will do all the work, but could you please tell me what I have to do? I have already contributed to couchbase, so I know how to use gerrit.

[abhinavdangeti]

@jgschis Go ahead and raise a pull request adding your code to this project.
There's instructions for it here.

[jgschis]

Hi,

I am not changing bleve yet. First I need to add my plural stemmer repository to https://github.com/blevesearch. How do I do that?

Once it's added, I will make a pull request to update bleve so that it calls the code in the pluralstem project.

1. Add https://github.com/jgschis/pluralstem to https://github.com/blevesearch
2. Make https://github.com/blevesearch/bleve call https://github.com/jgschis/pluralstem

[abhinavdangeti]

@jgschis I've ported your code in with this PR: #1808

[jgschis]

Thanks