Skip to content

Commit

Permalink
fix docs test
Browse files Browse the repository at this point in the history
  • Loading branch information
pabloem committed Nov 5, 2020
1 parent 99a3372 commit da42aba
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions sdks/python/apache_beam/io/gcp/bigquery.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,16 +82,16 @@
ReadAllFromBigQuery
-------------------
Beam 2.27.0 introduces a new transform called `ReadAllFromBigQuery` which
allows you to define table and query reads from BigQuery at pipeline runtime.
allows you to define table and query reads from BigQuery at pipeline
runtime.:::
read_requests = p | beam.Create([
ReadFromBigQueryRequest(query='SELECT * FROM mydataset.mytable'),
ReadFromBigQueryRequest(table='myproject.mydataset.mytable')])
results = read_requests | ReadAllFromBigQuery()
A good application for this transform is in streaming pipelines to
refresh a side input coming from BigQuery. This would work like so:
refresh a side input coming from BigQuery. This would work like so:::
side_input = (
p
Expand All @@ -100,20 +100,20 @@
| 'MapToReadRequest' >> beam.Map(
lambda x: BigQueryReadRequest(table='dataset.table'))
| beam.io.ReadAllFromBigQuery())
main_input = (
p
| 'MpImpulse' >> beam.Create(sample_main_input_elements)
|
'MapMpToTimestamped' >> beam.Map(lambda src: TimestampedValue(src, src))
| 'WindowMpInto' >> beam.WindowInto(
window.FixedWindows(main_input_windowing_interval)))
result = (
main_input
| 'ApplyCrossJoin' >> beam.FlatMap(
cross_join, rights=beam.pvalue.AsIter(side_input)))
**Note**: This transform is supported on Portable runners only.
Writing Data to BigQuery
========================
Expand Down

0 comments on commit da42aba

Please sign in to comment.