Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SAMZA-2055: Async high level api #905

Closed
wants to merge 13 commits into from
Closed

SAMZA-2055: Async high level api #905

wants to merge 13 commits into from

Conversation

mynameborat
Copy link
Contributor

@mynameborat mynameborat commented Feb 5, 2019 •
edited
Loading

vjagadish1989
vjagadish1989 suggested changes Mar 6, 2019
View reviewed changes
Copy link
Contributor

@vjagadish1989 vjagadish1989 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, @bharathkk!

samza-api/src/main/java/org/apache/samza/operators/functions/AsyncFlatMapFunction.java Show resolved Hide resolved
samza-api/src/main/java/org/apache/samza/operators/functions/AsyncFlatMapFunction.java Outdated Show resolved Hide resolved
samza-api/src/main/java/org/apache/samza/operators/functions/AsyncFlatMapFunction.java Outdated Show resolved Hide resolved
samza-core/src/main/java/org/apache/samza/task/StreamOperatorTask.java Outdated Show resolved Hide resolved
samza-core/src/main/java/org/apache/samza/task/StreamOperatorTask.java Outdated Show resolved Hide resolved
samza-core/src/main/java/org/apache/samza/operators/spec/AsyncOperatorSpec.java Outdated Show resolved Hide resolved
samza-api/src/main/java/org/apache/samza/operators/MessageStream.java Outdated Show resolved Hide resolved
samza-core/src/main/java/org/apache/samza/operators/impl/AsyncStreamOperatorImpl.java Outdated Show resolved Hide resolved
samza-core/src/main/java/org/apache/samza/operators/impl/OperatorImpl.java Outdated Show resolved Hide resolved
samza-core/src/main/java/org/apache/samza/operators/impl/AsyncStreamOperatorImpl.java Outdated Show resolved Hide resolved
@vjagadish1989
Copy link
Contributor

  1. Can you add an e2e integration test for a StreamApp + async-api
  2. Additionally, update our website-docs and add public code examples for usage

@vjagadish1989
Copy link
Contributor

vjagadish1989 commented Mar 8, 2019 •
edited
Loading

also adding @xinyuiscool in case he has additional feedback on the async-api

@mynameborat mynameborat changed the title SAMZA-2055: [WIP] Async high level api SAMZA-2055: Async high level api Mar 19, 2019
@mynameborat
Copy link
Contributor Author

  • Address review comments and renamed StreamOperatorImpl to FlatmapOperatorImpl
  • Added sample AsyncApplicationExample to illustrate the async APIs
  • Added end to end integration tests
  • Move the common data models used in the sample application into its own package for reusability

@vjagadish1989 please take a look

@mynameborat
Copy link
Contributor Author

testSinkEndToEndWithKeyWithNullRecords seems to be flaky. I have a 60% pass rate locally.
image

Following up with @srinipunuru to debug it

vjagadish1989
vjagadish1989 approved these changes Apr 2, 2019
View reviewed changes
Copy link
Contributor

@vjagadish1989 vjagadish1989 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ship it!

@asfgit asfgit closed this in 1b267f7 Apr 3, 2019
@mynameborat mynameborat deleted the async-high-level-api branch March 7, 2020 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vjagadish1989 vjagadish1989 vjagadish1989 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mynameborat @vjagadish1989