lunr-languages?

[juju2143]

The README.md mentions using elasticlunr.js to index other language documents with lunr-languages, but the repository seems gone now. Can I still index, say, French documents, or it's only English-only for now?

[weixsong]

https://github.com/MihaiValentin/lunr-languages

[weixsong]

I will update the link.

[juju2143]

Thanks for the answer, but unfortunately, it gives an error about lunr.SortedSet when I try it.

lunr.fr.js:688
    lunr.fr.stopWordFilter.stopWords = new lunr.SortedSet();
                                       ^

TypeError: lunr.SortedSet is not a function
    at /home/juju/mlpsubs/lunr.fr.js:688:40
    at Object.<anonymous> (/home/juju/mlpsubs/serialize.js:8:24)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:134:18)
    at node.js:962:3

Here's my node.js code:

var lunr = require('elasticlunr');

require('./lunr.stemmer.support.js')(lunr);
require('./lunr.fr.js')(lunr);

var index = lunr(function() {
  this.use(lunr.fr);
  this.addField('text');
  this.setRef('id');
  this.saveDocument(false);
});

Am I doing it right or that's a legitimate error?

[Kreozot]

Thanks!

[Kreozot]

I think it would make sense to cherry pick this from lunr.js to improve compatibility.

I think it needs some refactoring. Because it uses global lunr object which doesn't exist in elasticlunr implementation.

[Kreozot]

@weixsong If I prepare this PR, will you merge it?

[weixsong]

Hi, @Kreozot , I did not use elasticlunr.js with other languages so I did not notice that it does not work with lunr-languages, and I have checked the code, it did need some refactor to support other languages to make elasticlunr.js more powerful.

I will try to refactor the code to provide other language support. But if you could do that is good. But my idea is refactor totally to add other language support, that need to design a abstract interface that for function such as tokenizer, stemmer, etc. I dont want to just make a copy of Sorted_set in lunr.js, actually in the beginning I removed Sorted_set.js just because the performance of it and it's useless in elasticlunr.js.

:)

[Kreozot]

Wow, it will be cool! Good luck with that!

So, for now it is possible to using lunr-languages with elasticlunr in node.js with this code (it needs to have lunr and lunr-languages in your deps, but do not need to edit any code):

var elasticlunr = require('elasticlunr');
global.lunr = elasticlunr;
require('lunr/lib/sorted_set.js');
require('lunr-languages/lunr.stemmer.support.js')(elasticlunr);
require('lunr-languages/lunr.ru.js')(elasticlunr);

[weixsong]

Lunr-languages.js is now supported both in browser & Node.js, please try:

Other Languages

Default supported language of elasticlunr.js is English, if you want to use elasticlunr.js to index other language documents, then you need to use elasticlunr.js combined with lunr-languages.

Other languages example in Browser

Suppose you are using elasticlunr.js in browser for other languages, you could download the corresponding language support from lunr-languages, then include the scripts as:

<script src="lunr.stemmer.support.js"></script>
<script src="lunr.de.js"></script>

then, you could use elasticlunr.js as normal:

var index = elasticlunr(function () {
    // use the language (de)
    this.use(elasticlunr.de);
    // then, the normal elasticlunr index initialization
    this.addField('title')
    this.addField('body')
});

Pay attention to the special code:

    this.use(elasticlunr.de);

If you are using other language, such as es(Spanish), download the corresponding lunr.es.js file and lunr.stemmer.support.js, and change the above line to:

    this.use(elasticlunr.es);

Other languages example in Node.js

Suppose you are using elasticlunr.js in Node.js for other languages, you could download the corresponding language support from lunr-languages, put the files lunr.es.js file and lunr.stemmer.support.js in your project, then in your Node.js module, use elasticlunr.js as:

var elasticlunr = require('elasticlunr');
require('./lunr.stemmer.support.js')(elasticlunr);
require('./lunr.de.js')(elasticlunr);

var index = elasticlunr(function () {
    // use the language (de)
    this.use(elasticlunr.de);
    // then, the normal elasticlunr index initialization
    this.addField('title')
    this.addField('body')
});

For more details, please go to lunr-languages.

[biodranik]

@weixsong Is it possible to support multi-language feature? When the text contains strings in two different languages, e.g. Russian and English?

[stevedevhere]

@biodranik

@weixsong Is it possible to support multi-language feature? When the text contains strings in two different languages, e.g. Russian and English?

Hi, have you solved this question? I also need to do a search where the article contains Russian and English text