Fixes to FlsaModel

[ERijck]

Remove fuzzytm dependency and add ImportError for pyfume. Fixes points raised in #3428.

[osma]

Would it be worth declaring pyfume as an optional dependency of flsamodel? That would abstract the relationship a bit further. So basically use extras_require in setup.py like this:

    extras_require={
        "flsamodel": ["pyfume==0.2.5"],
    },

Then change the user instructions in the error message (and any related documentation) to this:

pip install gensim[flsamodel]

This would abstract away the specifics - maybe in a future release, the pyfume version requirement will be bumped, or another dependency is needed for flsamodel, but the command to install the needed dependencies remains the same and the details will be managed in setup.py just like required dependencies.

[piskvorky]

I like that idea!

@ERijck is really struggling with git though. Would you be able to gather all these changes in a single PR @osma? So we could finally merge & release a bugfix. Thanks!

[osma]

I can help, but I don't want to step on anyone's toes, nor take credit for other people's work. Also, learning opportunities shouldn't be wasted!

How about this: If this PR is otherwise OK from your perspective, then merge it as it is now. I will then do (today) a subsequent PR only for the optional dependency declaration. After you have merged both PRs, you can make a release. That way @ERijck gets all the due credit and my little adjustment comes on top of that.

[piskvorky]

Unfortunately it's not OK – there is a conflicting PR at #3436, attempting (but failing) another one-line change.

Learning opportunities are fine but we need to get the bugfix out ASAP. You could include @ERijck 's commits (~by way of credit), that's not a problem.

[piskvorky]

Actually, I have a bit of time today so I'll do a minor code clean-up and push a PR myself :)

[ERijck]

That works too! Thank you for your understanding and patience. I will make sure to understand the git workflow better before I make my next PRs.

[osma]

I see, I wasn't aware of the other, conflicting PR. Well, I hope you can get it all sorted out, and if it helps, I can do a PR with the optional dependency part (though it really just adding the few lines I posted in the above comment to setup.py)

[piskvorky]

@ERijck do you have a reference / link to an academic paper for FlsaModel? Something online where people can read more about what it is, how it works?

The docstring says only Fuzzy topic models with methods similar to LdaModel which is not terribly illuminating.

[piskvorky]

Similar with the parameters:

            cluster_method: str ['fcm', 'gk', 'fst-pso']
                Fuzzy clustering method.

Is there a description / tutorial somewhere for what this means? Which value to choose when and why?

[ERijck]

@piskvorky, I will write a post on my Towards Data Science account for FLSAmodel this weekend.

Furthermore, see the links to the algorithm papers below.

FLSA-W
https://ieeexplore.ieee.org/abstract/document/9660139

FLSA
https://link.springer.com/article/10.1007/s40815-017-0327-9

FLSA-E
https://ieeexplore.ieee.org/abstract/document/9945594?casa_token=Qb2ldPDBMO8AAAAA:R7If1JpDL8o8TrNS9yl9hNVXYcI05CccvRgDcAFDlevPMlRlvQqL6zDhvsHl6bvHJsfeOWiqVKk

And links to the fuzzy clustering algorithms below:

FCM
https://www.sciencedirect.com/science/article/pii/0098300484900207

GK
https://ieeexplore.ieee.org/abstract/document/4046215

FST-PSO
https://www.sciencedirect.com/science/article/pii/S2210650216303534

[piskvorky]

Continuing further down the rabbit hole: Am I reading this correctly (_convert_bow) that FlsaModel doesn't support streaming input, and works strictly over "list of list of strings"?

That is at odds with Gensim's core mission and interfaces.

[piskvorky]

The code requires much more work than I anticipated. I opened #3437 which outlines what needs to be done. As it stands now, I'm -1 on keeping FlsaModel in Gensim, this should not have been merged.

@mpenkov as a stop-gap measure, I propose a minimal PR to drop the setup.py requirement + bugfix.
Then if @ERijck manages to finish the fixes in time for the next release, we'll keep Flsa. Otherwise I propose dropping the model. I know, unfortunate, but the current code quality is insufficient and would give headaches to both Gensim maintainers and users.