-
Notifications
You must be signed in to change notification settings - Fork 281
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Google Cloud PubSub support #48
Conversation
cb443c8
to
34d2289
Compare
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
to address the issue in grpc where `-D_GLIBCXX_USE_CXX11_ABI=0` is not passed automatically Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
/cc @terrytangyuan This PR is also ready. The build is enabled in macOS and Linux. The test is done in Linux (skipped on macOS for now as there seems to be a bug in google SDK's macOS support?). Please take a look |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Though I wonder where we should keep the compatibility matrix, e.g. some ops don't work on certain system, etc.
@@ -29,8 +29,7 @@ if [[ "$(uname)" == "Darwin" ]]; then | |||
(cd kafka_2.11-0.10.1.0/ && bin/kafka-server-start.sh -daemon config/server.properties) | |||
echo -e "D0\nD1\nD2\nD3\nD4\nD5\nD6\nD7\nD8\nD9" > kafka_2.11-0.10.1.0/test | |||
echo Wait 15 secs until kafka is up and running | |||
# Note somehow it takes longer time in Darwin | |||
sleep 15 | |||
sleep 5 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't take longer anymore in Darwin?
Add Google Cloud PubSub support
This PR is part of the effort to support Google Cloud PubSub, which is a vendor specific cloud streaming platform on GCP. The concept of PubSub is similar to Apache Kafka.
Google Cloud PubSub does not have a C++ SDK, though it has a gRPC endpoint so this implementation is based on gRPC.
This implementation is tested with google cloud pubsub emulator for now. Will need additional effort to bring GCP IAM support in place.
This PR fixes #31.