Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cascading tap needs to be able to write to multiple indexes? #51

Closed
jeroenvandijk opened this issue Jun 4, 2013 · 3 comments
Closed

Cascading tap needs to be able to write to multiple indexes? #51

jeroenvandijk opened this issue Jun 4, 2013 · 3 comments
Labels
:Cascading v1.3.0.M3

Comments

@jeroenvandijk
Copy link

I'm trying to export some of my data to ES for Kibana. This now requires indexes per day. However AFAIK this is currently not possible in an easy way with the ES Cascading Tap since you have to choose an index upfront.

I think the ES tap needs a possibility to define a pattern for the index so it is possible to dynamically write to an index. I guess I need something similar as the GlobHfs Tap

What do you think?
@costin
Copy link
Member

costin commented Jun 10, 2013

Good idea - I'll take a look at it and see how much of GlobHfs can be used.

@kimchy
Copy link
Member

kimchy commented Jun 10, 2013

I think we should have this available in all outputs, i.e. use an index pattern, like myIndex-YYYYMMdd, and derive the YYYYMMdd based on the data indexed (timestamp for example).

@costin costin mentioned this issue Aug 14, 2013
Closed
@costin costin added v1.3.0.M3 and removed v1.3.0.M2 labels Feb 6, 2014
@costin
Copy link
Member

costin commented Apr 8, 2014

This is now supported in the upcoming 1.3 M3 without the need of an extra Glob/Tap. Timestamp formatting is on the roadmap for 1.3 RC1.

@costin costin closed this as completed Apr 8, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Cascading v1.3.0.M3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jeroenvandijk @kimchy @costin