Fix TermGenerator do not stop stemmed term.

[gauravaror]

It seems that the TermGenerator? (with STOP_ALL and STEM_ALL strategy) do not stop stemmed term.

For example, with french stemmer and "le" in the stopwords, le term "lea" will be stemmed to "le" and "le" term will be added to the document.

Looking into termgenerator_internal.cc (method index_text), it seems that the stopper in never called on the stem, whatever the flags are.

We are using version 1.4.2 but the stopper is never called on the stem even on git master.

Fixed the issue by changing code and added a test case to fix this issue.

[gauravaror]

Ah! It picked up old perl pod commits aswell i thought i created branch from the master but seems like i branched from another place.

Shall i recreate the branch?

[gauravaror]

Opened a new pull request with only this change

#174

[ojwb]

There's really no need to open a new PR in this situation - you can just rebase the branch onto master and then force push it to update the existing PR.

Also, you really should look over the diff that's shown before confirming you want to create the PR.

We rightly moan at the GSoC students for this sort of thing!

[gauravaror]

The two extra commit was not in master but in some other branch. I was not sure how would rebasing the branch onto master will fix that. Since it will still have those two extra commits from other PR i created.

I think i rebased the branch onto master.

The only way I saw was to revert those commits. Which was unnecessarily 4 commits added two earlier and two to revert that commit's from the branch. Hence decided to make the new PR.

yeah! my bad. I think I was bit distracted while creating the PR and thought those two commits from different PR was merged and forgot why to wonder why they were here. Will try to not have this issue furtheron. Sorry

[jaylett]

@samuelharden if you'd done an interactive rebase onto master, you could have dropped the surplus commits then force pushed onto the PR branch.

[gauravaror]

oops true!!