-
Notifications
You must be signed in to change notification settings - Fork 4.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BEAM-53] Pub/sub client with gRPC implementation #120
Conversation
R: @kennknowles |
import java.util.Collection; | ||
|
||
/** | ||
* A helper interface for talking to pub/sub via an underlying transport. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the official name is "Pubsub", no slash, capital P, lowercase 's'.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
/** | ||
* Construct a new Pubsub grpc client. It should be closed via {@link #close} in order | ||
* to ensure tidy cleanup of underlying netty resources. (Or use the try-with-resources | ||
* construct since this class is {@link AutoCloseable}. If non-{@literal null}, use |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
close paren
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
Looks like checkstyle dislikes your copyright headers. |
Could do with a squash but this addresses your comments and supports some quirks needed later by the dataflow runner. |
Looking at the misc support code in PubsubIO thinks would be much clearer if we also had a PubsubApiaryClient and all the regex and other misc handling in PubsubIO was moved into PubsubClient. BUT I'd like to make steady progress, so how about we take that as a TODO? |
LGTM. Merging. |
/** | ||
* A helper interface for talking to Pubsub via an underlying transport. | ||
*/ | ||
public interface PubsubClient extends AutoCloseable { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It occurs to me that this PR would really benefit from a follow-up with an in-memory testing fake, with a test suite that can be applied to both it and the gRPC implementation.
I think this PR might not have received the attention it deserves:
Since this is IO-related, @dhalperi would be a really good reviewer. @mshields822, can I ask you to get these things fixed up? |
|
||
package com.google.cloud.dataflow.sdk.io; | ||
|
||
import com.google.api.client.repackaged.com.google.common.base.Preconditions; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Whoops
Package-private would be better than |
No worries:
|
Also, the extra deps (beyond grpc-pubsub-v1) are all needed by the gRPC client. They are also included by the grpc-pubsub-v1 dep, but our transitive dependency check requires them to be included directly. You can find the same deps inside the bigtable-client-core pom. So the only new dep here is on grpc-pubsub-v1 itself. |
This work continues in |
Support pub/sub via gRPC.
Will later use for pure Java pub/sub source/sink.