Add reindex method to index class #473

Closed
Tobion opened this Issue Oct 8, 2013 · 8 comments

Projects

None yet

6 participants

Collaborator
Tobion commented Oct 8, 2013

There should probably be a method in reindex all documents from one index into a new index. This is needed when one wants to change the mapping with no downtime. See http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/

Several other client libraries already include that functionality:

Early draft. For some reason I get a Java exception inside elastic search as soon as there are documents in the old index.

[Elastica\Exception\ResponseException]
  MapperParsingException[mapping [product]]; nested: ClassCastException[java.
  util.ArrayList cannot be cast to java.util.Map];

The method is missing arguments to optionally overwrite the mapping and settings for the new index. Otherwise it would just be a copy.

// Elastica\Index

    /**
     *
     *
     * @return string The new index name
     *
     * @see http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/
     */
    public function reindex()
    {
        $scrollResponse = $this->request('_search', Request::GET,
            array('query' => array(
                'match_all' => array()
            )),
            array('search_type' => 'scan', 'scroll' => '10m', 'size' => '100')
        );
        //var_dump($scrollResponse);

        $scrollId = $scrollResponse->getScrollId();
        unset($scrollResponse);

        $mapping = $this->getMapping();
        $settings = $this->getSettings()->get();

        $config = array();
        $config['mappings'] = $mapping[$this->_name];
        $config['settings'] = $settings;

        $this->_name = $this->_name . date('-Y-m-d-Gis');

        $this->create($config);

        unset($mapping, $settings, $config);

        $scrollResponse = $this->getClient()->request('_search/scroll', Request::GET,
            array(),
            array('scroll' => '10m', 'scroll_id' => $scrollId)
        );
        //var_dump($scrollResponse);

        return $this->_name;
    }
Owner
ruflin commented Oct 10, 2013

My suggestion is that this is something that should go into Elastica\Util as it is not directly an "index" function that can be called in Elasticsearch but a specific implementation. You pass the old and new index name and the magic is going to happen. What do you think?

@Tobion, did you complete your function to provide reindex functionality to Elastica?

I am looking at possibly doing the same thing, but came across your post and wondered if you had got any further with it?

Thanks, Russell

Owner
ruflin commented Jan 14, 2014

@russellseymour As far as I know it is not in Elastica yet. So it would be very nice if you could pick it up.

Owner
ruflin commented Mar 27, 2014

@Tobion Any updates?

@ruflin ruflin added the enhancement label Mar 27, 2014

+1

We just added reindexing at work using scan and scroll, I'll spend some time to extract it into Elastica\Util and make a pull request :)

Owner
ruflin commented Apr 14, 2015

@michellesanver Looking forward to it.

Contributor

Added in #853
Thanks @Tobion

@webdevsHub webdevsHub closed this May 29, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment