PoC : Search

[tcitworld]

How-to :

• Launch elasticsearch (tested with version 1.74)
• composer update
• php bin/console fos:elastica:populate
• add entries
• search for entries
• profit

TODO :

• a parameter to enable/disable this feature (for those who can't install ES)
• Make things properly
• use RabbitMQ to index without loss of perfs
• tests
• add ES in docker-compose

Future :

• find a way to display searched tags
• search in comments too

Should fix #18

[j0k3r]

I think you don't need the first \

[nicosomb]

Nice :)

I added 2 TODO: a parameter to disable this feature and changes in docker-compose.

[j0k3r]

And I guess that if we have questions about ES, we can ask for an expert 👋 @damienalexandre

[nicosomb]

Just read this blog post http://blog.zenika.com/2016/02/15/consolider-les-logs-docker-dans-un-elk/ you can find configuration for your docker-compose file.

[damienalexandre]

The french_stem token filter does not exist, you don't have any stemming on this analyzer.

I suggest you use elision too, instead of stop_fr. Have a look at this example.

Also, why tokenizing for french only? I'm not sure the contents are mono-lang?

[tcitworld]

Also, why tokenizing for french only? I'm not sure the contents are mono-lang?

Not at all. It was to test results. Should we put analysers for all languages or is there an other way ?

[damienalexandre]

There is no way you will be able to get a good analysis if you don't know the language of the content, so my best guess would be to build a triGram analyzer, with html_strip char filter etc... and use multi-fields.

Each "searchable field" could be mapped:

• with a triGram analyzer (to collect a lot of contents)
• with standard analyzer (to improve the pertinence)

It will not be perfect 😞 and will need some tuning when it's setup (there are so many ways to do analysis, you have to found one that will fit your contents and how you want to search them).

[tcitworld]

The library we use can sometime provide us the language content. Can it be helpful too ?

[damienalexandre]

It can help yes,
you could chose to support some strong elision on a list of language (the one already in ES core are a good starting point), and search on those on a content basis.

Have a quick look at this to learn more about the recommended config: https://www.elastic.co/guide/en/elasticsearch/guide/current/mixed-lang-fields.html#_use_n_grams

[tcitworld]

Strange error.

Wallabag\CoreBundle\Tests\Controller\EntryControllerTest::testQuickstart
InvalidArgumentException: Unreachable field "entry"
Wallabag/CoreBundle/Tests/Controller/EntryControllerTest.php:44

[damienalexandre]

This should be part of the "mappings"; why is it here?