Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more information in Summarizer fromV #49

Closed
soloman817 opened this issue Aug 20, 2018 · 3 comments
Closed

add more information in Summarizer fromV #49

soloman817 opened this issue Aug 20, 2018 · 3 comments

Comments

@soloman817
Copy link

In this interface: https://github.com/twosigma/flint/blob/master/src/main/scala/com/twosigma/flint/timeseries/summarize/Summarizer.scala#L211

Sometime it is also good to know on which row it is rending. Is it possible to have something like: def fromV(v: V, t: T): InternalRow

@icexelloss
Copy link
Member

fromV() should be only called once after all rows are summarized, so there is no particular row associated with it. Can you elaborate why do you think it should be associate with an input row?

@soloman817
Copy link
Author

Hi thanks for the reply, I think I figured it out now, it is not needed. The reason is, in https://github.com/twosigma/flint/blob/master/src/main/scala/com/twosigma/flint/rdd/function/window/SummarizeWindows.scala#L833 , the state is stored as key -> state, where the key is constructed by the time and the key. So you actually will be able to know which row you are rendering if you store them during state creating in the add function.

@soloman817
Copy link
Author

Forgot to mention, I was using summarizer in a window summarization, so the render function will be called on each row, to generate results for that row.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants