Add Stemmer tests

[mateonunez]

Add stemmer tests. This PR closes #18 and improves the coverage.

Current (main) coverage:

 PASS  packages/lyra/tests/trie.test.ts
 PASS  packages/lyra/tests/utils.test.ts
 PASS  packages/lyra/tests/tokenizer.test.ts
 PASS  packages/lyra/tests/lyra.test.ts
-----------------|---------|----------|---------|---------|---------------------
File             | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s   
-----------------|---------|----------|---------|---------|---------------------
All files        |   91.97 |    79.34 |      90 |   93.28 |                     
 src             |   90.43 |    76.78 |   84.61 |   91.32 |                     
  errors.ts      |    82.6 |      100 |      50 |      75 | 8,20,23,26          
  levenshtein.ts |   91.66 |       60 |     100 |     100 | 2-3                 
  lyra.ts        |   93.69 |    78.78 |     100 |   94.54 | 127,164-167,182,193 
  stemmer.ts     |   76.92 |    72.72 |     100 |   76.92 | 22-23,43-44,49,56   
  tokenizer.ts   |     100 |      100 |     100 |     100 |                     
  utils.ts       |   94.73 |    83.33 |     100 |   94.73 | 30                  
 src/prefix-tree |   95.55 |    83.33 |     100 |    97.7 |                     
  node.ts        |     100 |      100 |     100 |     100 |                     
  trie.ts        |   94.59 |    83.33 |     100 |   97.18 | 121,146             
-----------------|---------|----------|---------|---------|---------------------

Coverage after the PR:

 PASS  packages/lyra/tests/utils.test.ts
 PASS  packages/lyra/tests/trie.test.ts
 PASS  packages/lyra/tests/stemmer.test.ts
 PASS  packages/lyra/tests/tokenizer.test.ts
 PASS  packages/lyra/tests/lyra.test.ts
-----------------|---------|----------|---------|---------|---------------------
File             | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s   
-----------------|---------|----------|---------|---------|---------------------
All files        |   93.31 |    81.31 |   90.24 |   94.68 |                     
 src             |   92.34 |       80 |   85.18 |   93.33 |                     
  errors.ts      |    82.6 |      100 |      50 |      75 | 8,20,23,26          
  levenshtein.ts |   91.66 |       60 |     100 |     100 | 2-3                 
  lyra.ts        |   93.69 |    78.78 |     100 |   94.54 | 127,164-167,182,193 
  stemmer.ts     |    92.3 |       90 |     100 |      92 | 49-50               
  tokenizer.ts   |     100 |      100 |     100 |     100 |                     
  utils.ts       |   94.73 |    83.33 |     100 |   94.73 | 30                  
 src/prefix-tree |   95.55 |    83.33 |     100 |    97.7 |                     
  node.ts        |     100 |      100 |     100 |     100 |                     
  trie.ts        |   94.59 |    83.33 |     100 |   97.18 | 121,146             
-----------------|---------|----------|---------|---------|---------------------

[micheleriva]

Thank you so much! I just fear you need to fetch upstream, as I already added some tests for the stemmer in a previous commit 🙂

[mateonunez]

I think I did it here b32b138 🧐

Should I move stemmer tests into tokenizer tests?

[micheleriva]

Oh, sorry! So, given that the stemmer has a dependency on the tokenizer, we can move all the tests into the tokenizer tests for now