Skip to content
This repository was archived by the owner on Feb 20, 2019. It is now read-only.

Port Django management command from Kitsune and Kuma.#168

Closed
jezdez wants to merge 13 commits into
mozilla:masterfrom
jezdez:commands
Closed

Port Django management command from Kitsune and Kuma.#168
jezdez wants to merge 13 commits into
mozilla:masterfrom
jezdez:commands

Conversation

@jezdez

@jezdez jezdez commented Sep 10, 2013

Copy link
Copy Markdown
Contributor

This is work in progress since I need to port over some tests, too ;)

@willkg

willkg commented Sep 10, 2013

Copy link
Copy Markdown
Member

Wow! I'm pretty psyched about this. I skimmed it and it looks good so far. I'm pretty hard up for free time, but after you land some tests and docs, I'll make a point of making time to look through it more carefully.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another optimization could be to set num_replicas to zero while re-indexing, which reduces copying data between nodes. Once you're done you can set it back to whatever it was and ES will bulk copy to the replication nodes.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Will incorporate this :)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robhudson Do you happen to have a reference somewhere that this is supposed to be used like this?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've tried to find docs on it but couldn't. If you have enough data locally you could maybe test it both ways and see which is faster. I forget where I learned of this... either the Elasticsearch training or maybe from Hanno while I was writing the reindexing jobs for marketplace?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I remember hearing about this during ES training, so that's probably where you heard it too.

Comment thread docs/django.rst Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More specifically, it's a mapping of doctypes -> indexes to write to.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants