feat: Add a flushAll() method that will flush all the inflight request and make sure all responses returned

[yirutang]

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

• Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
• Ensure the tests and linter pass
• Code coverage does not decrease (if any source code was changed)
• Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> ☕️

[codecov]

Codecov Report

Merging #492 into master will increase coverage by 0.56%.
The diff coverage is 85.10%.

@@             Coverage Diff              @@
##             master     #492      +/-   ##
============================================
+ Coverage     78.11%   78.67%   +0.56%     
- Complexity      642      662      +20     
============================================
  Files            58       59       +1     
  Lines          3687     3789     +102     
  Branches        235      255      +20     
============================================
+ Hits           2880     2981     +101     
- Misses          687      692       +5     
+ Partials        120      116       -4     

Impacted Files	Coverage Δ	Complexity Δ
...google/cloud/bigquery/storage/v1alpha2/Waiter.java	57.14% <33.33%> (-0.69%)	14.00 <4.00> (+2.00)	⬇️
.../cloud/bigquery/storage/v1alpha2/StreamWriter.java	84.17% <92.68%> (+1.95%)	39.00 <4.00> (+4.00)
.../cloud/bigquery/storage/v1alpha2/DirectWriter.java	77.55% <0.00%> (-3.10%)	6.00% <0.00%> (+1.00%)	⬇️
...oud/bigquery/storage/v1alpha2/JsonWriterCache.java	87.93% <0.00%> (ø)	12.00% <0.00%> (?%)
...ud/bigquery/storage/v1alpha2/JsonStreamWriter.java	77.39% <0.00%> (+0.19%)	13.00% <0.00%> (+1.00%)
...ery/storage/v1beta1/BaseBigQueryStorageClient.java	60.34% <0.00%> (+1.72%)	19.00% <0.00%> (ø%)
...ud/bigquery/storage/v1/BaseBigQueryReadClient.java	43.90% <0.00%> (+2.43%)	10.00% <0.00%> (ø%)
...gquery/storage/v1beta2/BaseBigQueryReadClient.java	43.90% <0.00%> (+2.43%)	10.00% <0.00%> (ø%)
...bigquery/storage/v1alpha2/BigQueryWriteClient.java	58.57% <0.00%> (+5.71%)	28.00% <0.00%> (ø%)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28dace8...c6529a1. Read the comment docs.

[yirutang]

That's a design detail, a choice of either make flush wait or append wait. For now, let's keep it simple by asking append wait. In the final operation, user has to make sure flush is called last.

…

On Sat, Aug 15, 2020 at 9:51 AM weizhengli1992 ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In google-cloud-bigquerystorage/src/main/java/com/google/cloud/bigquery/storage/v1alpha2/StreamWriter.java <#492 (comment)> : > @@ -265,6 +273,28 @@ public Boolean expired() { return outstandingAppend.appendResult; } + /** + * This is the general flush method for asynchronise append operation. When you have outstanding + * append requests, calling flush will make sure all outstanding append requests completed and + * successful. Otherwise there will be an exception thrown. + * + * @throws Exception + */ + public void flushAll() throws Exception { + writeAllOutstanding(); + synchronized (messagesWaiter) { I had this discussion with Bigang previously and we decided to keep allow user to append instead of block. Just wait for all pending stream before the time user call FlushAll() but keep accepting new requests. (Flush all will not wait for new requests). Do you think this make sense to this case as well? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#492 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHGYVETCS7BHII7DNPCA3R3SA24HJANCNFSM4P74BZSQ> .

-- Thanks. Yiru