Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Create Stream writer v2 - starting with core logics #867

Merged
merged 4 commits into from Feb 23, 2021

Conversation

yayi-google
Copy link
Contributor

@yayi-google yayi-google commented Feb 21, 2021

Unblocks Dataflow sink test which unblocks customer update.

yayi-google added 2 commits Feb 21, 2021
Compared to existing StreamWriter, its locking mechanism is much
simpler.
@google-cla google-cla bot added the cla: yes label Feb 21, 2021
@product-auto-label product-auto-label bot added the api: bigquerystorage label Feb 21, 2021
@yayi-google yayi-google marked this pull request as ready for review Feb 21, 2021
@yayi-google yayi-google requested review from as code owners Feb 21, 2021
@codecov
Copy link

@codecov codecov bot commented Feb 22, 2021

Codecov Report

Merging #867 (eabad67) into master (9c103df) will increase coverage by 0.60%.
The diff coverage is 95.71%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #867      +/-   ##
============================================
+ Coverage     80.03%   80.64%   +0.60%     
- Complexity      979     1005      +26     
============================================
  Files            74       76       +2     
  Lines          5290     5430     +140     
  Branches        408      420      +12     
============================================
+ Hits           4234     4379     +145     
- Misses          872      878       +6     
+ Partials        184      173      -11     
Impacted Files Coverage Δ Complexity Δ
...cloud/bigquery/storage/v1beta2/StreamWriterV2.java 94.95% <94.95%> (ø) 23.00 <23.00> (?)
...oud/bigquery/storage/v1beta2/StreamConnection.java 100.00% <100.00%> (ø) 3.00 <3.00> (?)
...ery/storage/v1beta1/BaseBigQueryStorageClient.java 72.88% <0.00%> (+1.69%) 22.00% <0.00%> (ø%)
...ud/bigquery/storage/v1/BaseBigQueryReadClient.java 60.97% <0.00%> (+2.43%) 12.00% <0.00%> (ø%)
...gquery/storage/v1beta2/BaseBigQueryReadClient.java 60.97% <0.00%> (+2.43%) 12.00% <0.00%> (ø%)
.../bigquery/storage/v1beta2/BigQueryWriteClient.java 75.38% <0.00%> (+4.61%) 33.00% <0.00%> (ø%)
...bigquery/storage/v1alpha2/BigQueryWriteClient.java 80.51% <0.00%> (+6.49%) 38.00% <0.00%> (ø%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9c103df...b98d6be. Read the comment docs.

@yirutang yirutang self-requested a review Feb 23, 2021
log.info("User closing stream: " + streamName);
this.lock.lock();
try {
this.userClosed = true;
Copy link
Collaborator

@bigang-li bigang-li Feb 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding a local var to remember the previous value, so if user closed twice, the second time will be no_op (don't call the appendThread.join() multiple times).


if (localQueue.isEmpty()) {
continue;
}
Copy link
Collaborator

@bigang-li bigang-li Feb 23, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe be save a few lines of code here? it's useless according to context.

@stephaniewang526 stephaniewang526 merged commit 7c01f45 into googleapis:master Feb 23, 2021
15 of 16 checks passed
@yayi-google yayi-google deleted the stream-writer-v2 branch Feb 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquerystorage cla: yes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants