New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Future index preparation #46

Closed
jordansissel opened this Issue Feb 13, 2014 · 7 comments

Comments

Projects
None yet
4 participants
@jordansissel
Copy link
Contributor

jordansissel commented Feb 13, 2014

Proposed roughly from a discussion in #logstash IRC -

Create a future index with dynamic settings, such as doing shard allocation based on some computed value, etc.

@HonzaKral HonzaKral added enhancement and removed enhancement labels Feb 14, 2014

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Apr 4, 2014

the burst of new mappings has been heavily optimized in 1.0, I don't think this is curator job to fix.

@jordansissel

This comment has been minimized.

Copy link
Contributor

jordansissel commented Apr 4, 2014

@kimchy ok cool, I removed the mapping point. Thanks!

@jordansissel

This comment has been minimized.

Copy link
Contributor

jordansissel commented Apr 4, 2014

I moved the 'alias management' point to a new ticket.

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Apr 4, 2014

@jordansissel obviously, if its still a problem in 1.0.x, then we should heavily look into this again. The problem with copying mapping is that its tricky to provide for change easily...

@untergeek

This comment has been minimized.

Copy link
Member

untergeek commented Apr 4, 2014

@kimchy That's interesting to hear. I had no problems with 0.90.x on my teeny-tiny cluster. I had no problems with 1.0 on the same cluster. I'm having fits with 1.1 on this same cluster. Within 10 minutes after UTC rollover, my Logstash has stopped sending data for the last several days. After a restart all is well.

There are no indicators in either log file (though I'm admittedly not in debug for Logstash, but it didn't seem to matter there).

For some reason, it just stops cold after a bit in the new index. I can't figure out why. When Logstash is sending to disk, I have no such problems. Only when I send to ES (and it doesn't matter if it's http or node or transport protocol from my testing).

@untergeek

This comment has been minimized.

Copy link
Member

untergeek commented Apr 4, 2014

Btw, that's a Logstash restart, not an ES one. ES keeps going, but for some reason LS thinks it can't and stops.

@untergeek

This comment has been minimized.

Copy link
Member

untergeek commented Sep 25, 2014

This is largely moot with optimizations which have been added to Elasticsearch in the interim. Namely, alias and allocation assignment at index creation time (including creation from template).

@untergeek untergeek closed this Sep 25, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment