reuse this in scicloj.ml.smile ?

[behrica]

For reference:

https://github.com/scicloj/scicloj.ml.smile/blob/d70c7e3caff93935d05ab81ed6b2d1e4846ad42b/src/scicloj/ml/smile/nlp.clj#L281

If possible I would like to re-use this implementation.

I only made one, because I did not find any some months ago.

[behrica]

I had a quick look and 2 comments:

1. I was thinking to change my implementation and convert the "vocabulary" into a list of ints, instead of list of strings.
   Just hope to gain memory with this.
   Any comment on this ?

2. I implemented several "alternative" calculations.
   By either doing "the same as sklearn" and / or looking at all formulas here:
   https://en.wikipedia.org/wiki/Tf%E2%80%93idf
   For both tf and idf it lists several weighting schemes.

3. The current public API here looks good to me, for potentially plugin in into scicloj.ml.smile

4. is important as I noticed big differences in downstream uses cases of tf-idf (comparing documents in my case)

I have maybe 4 out of 10 implemented.

[simongray]

Hi @behrica, I've added an MIT licence, so you're welcome to use the code in any way that fits the licence. Basically, it just means that you need to maintain the copyright notice. :)