Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cascading tap needs to be able to write to multiple indexes? #51

Closed
jeroenvandijk opened this Issue Jun 4, 2013 · 3 comments

Comments

Projects
None yet
3 participants
@jeroenvandijk
Copy link

jeroenvandijk commented Jun 4, 2013

I'm trying to export some of my data to ES for Kibana. This now requires indexes per day. However AFAIK this is currently not possible in an easy way with the ES Cascading Tap since you have to choose an index upfront.

I think the ES tap needs a possibility to define a pattern for the index so it is possible to dynamically write to an index. I guess I need something similar as the GlobHfs Tap

What do you think?

@costin

This comment has been minimized.

Copy link
Member

costin commented Jun 10, 2013

Good idea - I'll take a look at it and see how much of GlobHfs can be used.

@kimchy

This comment has been minimized.

Copy link
Member

kimchy commented Jun 10, 2013

I think we should have this available in all outputs, i.e. use an index pattern, like myIndex-YYYYMMdd, and derive the YYYYMMdd based on the data indexed (timestamp for example).

@costin costin added v1.3.0.M3 and removed v1.3.0.M2 labels Feb 6, 2014

@costin costin added the cascading label Apr 8, 2014

@costin

This comment has been minimized.

Copy link
Member

costin commented Apr 8, 2014

This is now supported in the upcoming 1.3 M3 without the need of an extra Glob/Tap. Timestamp formatting is on the roadmap for 1.3 RC1.

@costin costin closed this Apr 8, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.