Skip to content

[Feature Request]: Set quota project in beam.io.ReadFromBigQuery #37431

@shahar1

Description

@shahar1

What would you like to happen?

recreation of #31126

This issue relates to the Python SDK, but it is probably relevant to other SDKs as well:
We have a use case where queries initiated by beam.io.ReadFromBigQuery should be billed on a specific GCP Project ID.
As we use a custom container, the only option for now would be setting the env. var. GOOGLE_CLOUD_QUOTA_PROJECT in the Dockerfile - but it affects all other GCP services as well.
It would be best making it configurable via the connector (i.e., beam.io.ReadFromBigQuery(..., quota_project_id='some-project-id)).
When implementing, you could gain inspiration from similar feature in beam.io.WriteToBigQuery: #16186.

Issue Priority

Priority: 2 (default / most feature requests should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions