Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(samples): add query external bigtable using temp table #763

Merged

Conversation

pmakani
Copy link
Contributor

@pmakani pmakani commented Sep 30, 2020

No description provided.

@pmakani pmakani requested a review from stephaniewang526 Sep 30, 2020
@google-cla google-cla bot added the cla: yes label Sep 30, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run label Sep 30, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Sep 30, 2020
@stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Sep 30, 2020

Hi @billyjacobson -- PTAL as well! Thank you.

@stephaniewang526 stephaniewang526 marked this pull request as ready for review Sep 30, 2020
@stephaniewang526 stephaniewang526 requested a review from Sep 30, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run label Sep 30, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Sep 30, 2020
@codecov
Copy link

@codecov codecov bot commented Sep 30, 2020

Codecov Report

Merging #763 into master will decrease coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #763      +/-   ##
============================================
- Coverage     80.40%   80.39%   -0.01%     
- Complexity     1263     1265       +2     
============================================
  Files            79       79              
  Lines          6517     6525       +8     
  Branches        740      743       +3     
============================================
+ Hits           5240     5246       +6     
- Misses          888      889       +1     
- Partials        389      390       +1     
Impacted Files Coverage Δ Complexity Δ
...main/java/com/google/cloud/bigquery/FieldList.java 100.00% <0.00%> (ø) 11.00% <0.00%> (ø%)
...google/cloud/bigquery/ExternalTableDefinition.java 58.55% <0.00%> (+0.46%) 21.00% <0.00%> (+2.00%)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4c5bdad...691519b. Read the comment docs.

queryExternalBigtableTemp(tableName, sourceUri, query);
}

public static void queryExternalBigtableTemp(String tableName, String sourceUri, String query) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have a sample schema for bigtable we've been using that would be good to include here so it feels a little more realistic. I'll send you a doc with the schema

table name: mobile-time-series
column families: stats_summary
columns: os_build (STRING), os_name (STRING)

Copy link
Member

@stephaniewang526 stephaniewang526 Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/googleapis/java-bigtable/blob/master/samples/snippets/src/test/java/com/example/bigtable/ReadsTest.java
Let's stick to the existing table schema as shown above instead of using our own schema and stream in data in the beforeClass() method. We can keep the table name as bigquery-samples-test

.setType("STRING")
.setEncoding("TEXT")
.build();
BigtableColumnFamily usStates =

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you define the column family first and then add the columns and build after? It will make more sense for someone from Bigtable following since column families are the higher level grouping

Copy link
Contributor Author

@pmakani pmakani Oct 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@billyjacobson It's not possible to main order, because first we need to create an object of BigtableColumn then we can set those columns in BigtableColumnFamily.

Copy link
Member

@stephaniewang526 stephaniewang526 Oct 2, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could first instantiate an empty BigtableColumnFamily object and then add columns to it.

@product-auto-label product-auto-label bot added api: bigquery samples labels Oct 1, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run label Oct 2, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Oct 2, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run label Oct 2, 2020
@pmakani pmakani requested review from as code owners Oct 5, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Oct 5, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run label Oct 5, 2020
@stephaniewang526
Copy link
Member

@stephaniewang526 stephaniewang526 commented Oct 5, 2020

LGTM -- @billyjacobson PTAL

@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Oct 5, 2020
String bigqueryTableName = "MY_TABLE_NAME";
String sourceUri =
String.format(
"https://googleapis.com/bigtable/projects/%s/instances/%s/tables/%s",
Copy link
Member

@stephaniewang526 stephaniewang526 Oct 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to be causing issue:

com.google.api.gax.rpc.NotFoundException: io.grpc.StatusRuntimeException: NOT_FOUND: Failed to read: projects/{779844219229}/instances/bigquery-samples-instance
Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: Failed to read: projects/{779844219229}/instances/bigquery-samples-instance ```

@stephaniewang526 stephaniewang526 added the kokoro:force-run label Oct 6, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Oct 6, 2020
@stephaniewang526 stephaniewang526 added the kokoro:force-run label Oct 6, 2020
@yoshi-kokoro yoshi-kokoro removed the kokoro:force-run label Oct 6, 2020
@stephaniewang526 stephaniewang526 merged commit fed28c7 into googleapis:master Oct 6, 2020
17 of 18 checks passed
@pmakani pmakani deleted the query-external-bigtable-temp branch Oct 7, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery cla: yes samples
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants