Multi type search to multi index search

[Kleinast]

Hello there,

The new elasticsearch don't allow to have more than one type by index.
So this doc is Obsolete, isn't it ?
https://github.com/FriendsOfSymfony/FOSElasticaBundle/blob/master/doc/cookbook/multi-type-search.md
I'am looking for do similar thing, but now with multi index search. How should I do ?

Thanks

[IckleChris]

Very new to ES and this bundle, but I've got as far as being able to perform a search across multiple indices;

indexes:
    thread:
        finder: ~
        types:
            thread:
                properties:
                    title: ~
                    body: ~
                persistence:
                    driver: orm
                    model: App\Entity\Thread
                    provider: ~
                    finder: ~
    reply:
        finder: ~
        types:
            reply:
                properties:
                    body: ~
                persistence:
                    driver: orm
                    model: App\Entity\Reply
                    provider: ~
                    finder: ~

$mngr = $this->get('fos_elastica.index_manager');

$search = $mngr->getIndex('thread')->createSearch($searchTerm);
$search->addIndex('reply');
$results = $search->search()->getResults();

$results contains matched items from both indexes. What I can't figure out is the transformation from that point.

$transformer = $this->get('fos_elastica.elastica_to_model_transformer.collection.thread');
$results = $transformer->transform($results);

In this case $transformer is an instance of ElasticaToModelTransformerCollection that (obviously) only contains the thread transformer, and therefore fails when it attempts to transform a reply. There may be an easy way to end up with a collection containing multiple transformers but I can't seem to find it.

[Kleinast]

Your solution work fine. Thank you very much

[insekticid]

@IckleChris:

full article here: https://www.exploit.cz/how-to-search-across-multiple-elasticsearch-indexes-with-symfony-fos-elasticabundle/

define service

    FOS\ElasticaBundle\Transformer\ElasticaToModelTransformerCollection:
        arguments:
            - {
                recipe: '@fos_elastica.elastica_to_model_transformer.recipe.recipe',
                recipes: '@fos_elastica.elastica_to_model_transformer.recipes.recipes'
              }

define custom class and inject this collection into $this->elasticaToModelTransformerCollection;

           $index      = $this->indexManager->getIndex('recipe');
           $index2     = $this->indexManager->getIndex('recipes');

           $searchable = new MultiIndex($index->getClient(), $index->getName());
           $searchable->addIndex($index);
           $searchable->addIndex($index2);

           $transformer = $this->elasticaToModelTransformerCollection;
           $paginatorAdapter = new TransformedPaginatorAdapter($searchable, $query, [], $transformer);
           $items = new Pagerfanta(new FantaPaginatorAdapter($paginatorAdapter));
           $items->setMaxPerPage($limit);
           $items->setCurrentPage($page);

where MultiIndex looks like

<?php

declare(strict_types=1);

/*
 * Created by Exploit.cz <insekticid@exploit.cz>
 */

namespace App\Search\Elastica;

use Elastica\Exception\InvalidException;
use Elastica\Index;
use Elastica\ResultSet\BuilderInterface;
use Elastica\Search;

class MultiIndex extends Index
{
    /**
     * Array of indices.
     *
     * @var array
     */
    protected $_indices = [];

    /**
     * Adds a index to the list.
     *
     * @param \Elastica\Index|string $index Index object or string
     *
     * @throws \Elastica\Exception\InvalidException
     *
     * @return $this
     */
    public function addIndex($index)
    {
        if ($index instanceof Index) {
            $index = $index->getName();
        }

        if (!is_scalar($index)) {
            throw new InvalidException('Invalid param type');
        }

        $this->_indices[] = (string) $index;

        return $this;
    }

    /**
     * Add array of indices at once.
     *
     * @param array $indices
     *
     * @return $this
     */
    public function addIndices(array $indices = [])
    {
        foreach ($indices as $index) {
            $this->addIndex($index);
        }

        return $this;
    }

    /**
     * Return array of indices.
     *
     * @return array List of index names
     */
    public function getIndices()
    {
        return $this->_indices;
    }

    /**
     * @param string|array|\Elastica\Query $query
     * @param int|array                    $options
     * @param BuilderInterface             $builder
     *
     * @return Search
     */
    public function createSearch($query = '', $options = null, BuilderInterface $builder = null)
    {
        $search = new Search($this->getClient(), $builder);

        $search->addIndices($this->getIndices());
        $search->setOptionsAndQuery($options, $query);

        return $search;
    }
}

[michaelKaefer]

The docs are still wrong for elasticsearch 6.0 and all following versions. Since this issue was closed: there is another open issue: #1521

[skylord123]

Yeah I just ran into this and there are no great alternatives.

It would be nice if this bundle got updated to handle multiple indexes in a better way.

I am building a global site search atm and indexes are only added if the user has permissions. Getting that to work along with pagination is pretty god awful right now.

[PaulAboulinc]

Very new to ES and this bundle, but I've got as far as being able to perform a search across multiple indices;

indexes:
    thread:
        finder: ~
        types:
            thread:
                properties:
                    title: ~
                    body: ~
                persistence:
                    driver: orm
                    model: App\Entity\Thread
                    provider: ~
                    finder: ~
    reply:
        finder: ~
        types:
            reply:
                properties:
                    body: ~
                persistence:
                    driver: orm
                    model: App\Entity\Reply
                    provider: ~
                    finder: ~

$mngr = $this->get('fos_elastica.index_manager');

$search = $mngr->getIndex('thread')->createSearch($searchTerm);
$search->addIndex('reply');
$results = $search->search()->getResults();

$results contains matched items from both indexes. What I can't figure out is the transformation from that point.

$transformer = $this->get('fos_elastica.elastica_to_model_transformer.collection.thread');
$results = $transformer->transform($results);

In this case $transformer is an instance of ElasticaToModelTransformerCollection that (obviously) only contains the thread transformer, and therefore fails when it attempts to transform a reply. There may be an easy way to end up with a collection containing multiple transformers but I can't seem to find it.

I used this response but I had problem with searching on other index than the one I used to get de transformer (thread in this example).

To avoid that, I override the class "ElasticaToModelTransformerCollection" like this :

<?php

namespace Unadere\Bundle\Utils\ElasticaBundle;

/**
 * Class ElasticaToModelTransformerCollection
 * @package Unadere\Bundle\Utils\ElasticaBundle
 */
class ElasticaToModelTransformerCollection extends \FOS\ElasticaBundle\Transformer\ElasticaToModelTransformerCollection
{
    /**
     * @return array|\FOS\ElasticaBundle\Transformer\ElasticaToModelTransformerInterface[]
     */
    public function getTransformers()
    {
        return $this->transformers;
    }

    /**
     * @param ElasticaToModelTransformerCollection $modelTransformer
     */
    public function addTransformers($modelTransformer)
    {
        $transformers = $modelTransformer->getTransformers();
        foreach ($transformers as $key => $transformer) {
            $this->transformers[$key] = $transformer;
        }
    }
}

In the OverrideServiceCompilerPass.php :

$definition = $container->findDefinition('fos_elastica.elastica_to_model_transformer.collection');
$definition->setClass(ElasticaToModelTransformerCollection::class);

Then in my service using the transformer, I create a new transformer :

Service arguments :

arguments:
   $searchTransformers:
         thread: '@fos_elastica.elastica_to_model_transformer.collection.thread'
         reply: '@fos_elastica.elastica_to_model_transformer.collection.reply'

And in my service :

public function __construct($searchTransformers)
    {
        $modelTransformer = new ElasticaBundle\ElasticaToModelTransformerCollection([]);
        /** @var \Unadere\Bundle\Utils\ElasticaBundle\ElasticaToModelTransformerCollection $transformer */
        foreach ($searchTransformers as $transformers) {
            $modelTransformer->addTransformers($transformers);
        }

        $this->modelTransformer = $modelTransformer;
    }

It's seems to work fine for me but I'm not against a better solution. :)

[opalenet-yann]

Hello,
You should be able to do the same without extending ElasticaToModelTransformerCollection
with a configuration like

    elastica_multiple_index_transformer:
        class: FOS\ElasticaBundle\Transformer\ElasticaToModelTransformerCollection
        arguments:
            - {'video': '@fos_elastica.elastica_to_model_transformer.video.video',
               'gallery': '@fos_elastica.elastica_to_model_transformer.gallery.gallery',
               'story': '@fos_elastica.elastica_to_model_transformer.story.story'
            }

[insekticid]

My Article updated, works well with latest master 6.0 Beta (need to create new ElasticaToModelTransformerCollection class)
https://www.exploit.cz/how-to-search-across-multiple-elasticsearch-indexes-with-symfony-fos-elasticabundle/

[skylord123]

@insekticid your post works pretty well but I updated a few things:

When defining the ElasticaToModelTransformerCollection you pass each transformer as an argument in an array. This is unnecessary because all transformers are tagged with fos_elastica.elastica_to_model_transformer so you can actually just specify it like this:

    App\Search\Transformer\ElasticaToModelTransformerCollection:
        arguments: [!tagged_iterator { tag: 'fos_elastica.elastica_to_model_transformer', index_by: 'index' }]

And the __construct() can be changed to

    public function __construct($transformers)
    {
        $this->transformers = iterator_to_array($transformers);
    }

And now all indexes get auto added automatically. I found it annoying that every time I added a new index I had to update my services.yaml file. This fixes that. I also added a custom transformer for one of my objects because this bundle doesn't handle ULID for id's correctly. When I did that fix I forgot to update the array of transformers which is another reason I ran into this annoyance.

Besides that good job and thanks. You saved me some time.

[matthiasseghers]

Currently having the same issue as well. Saw that the class was added to master, but was later removed in commit 926919195c2efaca2fb0b42304ca749780645295.
There does not seem to be any trace of BC in the upgrade doc.. So it's very unclear how to proceed. The only solution I can find is basically adding it back in like described in #1385. Is this the way to implement this? Or is there a different way to approach this migration of multi type -> multi index?