Port Django management command from Kitsune and Kuma.#168
Conversation
|
Wow! I'm pretty psyched about this. I skimmed it and it looks good so far. I'm pretty hard up for free time, but after you land some tests and docs, I'll make a point of making time to look through it more carefully. |
There was a problem hiding this comment.
Another optimization could be to set num_replicas to zero while re-indexing, which reduces copying data between nodes. Once you're done you can set it back to whatever it was and ES will bulk copy to the replication nodes.
There was a problem hiding this comment.
Nice! Will incorporate this :)
There was a problem hiding this comment.
@robhudson Do you happen to have a reference somewhere that this is supposed to be used like this?
There was a problem hiding this comment.
I've tried to find docs on it but couldn't. If you have enough data locally you could maybe test it both ways and see which is faster. I forget where I learned of this... either the Elasticsearch training or maybe from Hanno while I was writing the reindexing jobs for marketplace?
There was a problem hiding this comment.
I remember hearing about this during ES training, so that's probably where you heard it too.
Also wrap index setting in a try/finally block to make sure we set it back even if we raise an exception in between.
…index with the given name.
There was a problem hiding this comment.
More specifically, it's a mapping of doctypes -> indexes to write to.
This is work in progress since I need to port over some tests, too ;)