SAMZA-1268: Javadoc cleanup for public APIs for 0.13 release #169
Conversation
|
@vjagadish1989 @jmakes @bharathkk, please let me know if there's anything else we can clarify. |
| @@ -163,23 +169,25 @@ private Windows() { } | |||
| * | |||
| * <pre> {@code | |||
| * MessageStream<String> stream = ...; | |||
| * BiFunction<String, Integer, Integer> maxAggregator = (m, c)-> Math.max(parseInt(m), c); | |||
| * Supplier<Integer> initialValue = () -> 0; | |||
| * FoldLeftFunction<String, Integer, Integer> maxAggregator = (m, c) -> Math.max(parseInt(m), c); | |||
| * MessageStream<WindowPane<Void, Integer>> windowedStream = stream.window( | |||
| * Windows.tumblingWindow(Duration.ofSeconds(10), maxAggregator)); | |||
| * } | |||
| * </pre> | |||
| * | |||
| * @param duration the duration in processing time | |||
There was a problem hiding this comment.
Nit: Why is this parameter named differently for keyed and unkeyed tumbling windows?
| */ | ||
| @InterfaceStability.Unstable | ||
| public interface InitableFunction { | ||
|
|
||
| /** | ||
| * Interface method to initialize the context for a specific message transformation function. | ||
| * Initializes the function before any messages are processed. | ||
| * |
There was a problem hiding this comment.
Are there any guarantees we can provide about the order in which init() is invoked if there are multiple InitableFunctions in a StreamApplication?
It'd be nice to inform the users if so.
There was a problem hiding this comment.
Not currently. If there are multiple input streams, it'll depend on the iteration order for the Map that contains them.
That said, we should try to guarantee a deterministic and predictable iteration order if possible. Created SAMZA-1271 to track.
| * | ||
| * @param message the incoming message that is added to the window. This object should not be mutated. | ||
| * @param oldValue the previous value | ||
| * @param message the message being to the window |
| @@ -22,7 +22,8 @@ | |||
|
|
|||
|
|
|||
| /** | |||
| * A function that specifies whether a message should be retained for further processing or filtered out. | |||
| * Specifies whether a message should be retained for further processing. | |||
There was a problem hiding this comment.
Can a java Predicate impl be used interchangeably with this interface? If so, that'd be pretty powerful and worth mentioning.
Top of my head, we'd need this interface to extend Predicate, so the answer is probably no. Just throwing it out there.
There was a problem hiding this comment.
No, cannot be used interchangeably.
We discussed using java types for these functions (Predicate, Function, BiFunction etc.) instead of introducing our own earlier (see https://github.com/apache/samza/pull/51/files/001be632dcfdbbb138b792265b991722a2ea5597#diff-e1f7020353d9bd97ac274b6e3607d5e3 for more context). TL;DR currently these give us some flexibility to add marker interfaces or behavior (Initable, Closeable, Serializable) to these functions if we need it. For users that just use a lambda the difference shouldn't be noticeable.
I think we have a clearer expectations from these functions now, so we can discuss this again when @nickpan47's back.
| * emitted as matches are found. | ||
| * <p> | ||
| * <b>Note:</b> Currently messages in joins are kept in memory and may be lost in case of failures. |
There was a problem hiding this comment.
Might want to date this note with a release version
| @@ -100,6 +100,8 @@ | |||
| * {@link WindowPane}s. | |||
| * <p> | |||
| * Use the {@link org.apache.samza.operators.windows.Windows} helper methods to create the appropriate windows. | |||
| * <p> | |||
| * <b>Note:</b> Currently messages in windows are kept in memory and may be lost in case of failures. | |||
There was a problem hiding this comment.
Might want to date this note with a release version
| * {@link #sendTo(OutputStream)} instead. This transform should only be used to output to | ||
| * non-stream systems (e.g., an external database). | ||
| * <p> | ||
| * Offers more control over processing and sending messages than {@link #sendTo(OutputStream)} since |
There was a problem hiding this comment.
This is a very helpful note, thanks.
Might also be worth mentioning that this can also be used to send output on a channel that doesn't have a System impl
This PR cleans up javadocs for Fluent API classes in the samza-api module.
Also updates the TaskContext (existing) and ContextManager (new) interfaces to add support for setting an pass-through user-defined context.