BigQuery Data Transfer Config: Add Integer Partitioning Support

[kurtmaile]

Description

google_bigquery_data_transfer_config to support integer partitioning which is GA.

Seems this resource is generally getting left behind a bit of late, there are 4 outstanding issues on it, will it be given some priority? Realise the team will be busy on many things, but it is starting to force us to look away from TF on GCP.

New or Affected Resource(s)

• google_bigquery_data_transfer_config

Potential Terraform Configuration

resource "google_bigquery_data_transfer_config" "tc"  {

  display_name = "xxx"
  location = "EU"
  data_source_id = "scheduled_query"
  schedule = "xxxx"
  destination_dataset_id = "ds"
  params = {
    destination_table_name_template = "dt"
    write_disposition = "WRITE_TRUNCATE"
    query = "SELECT * FROM `v`"
    partitioning_field = "anInt"
    range_partitioning=customer_id,0,100,10
  }
}

This matches the 4 fields required for int bucket based partitioning.

References

https://cloud.google.com/bigquery/docs/creating-integer-range-partitions#integer_range_partitioning_with_clustering

b/284524817

[danawillow]

Hey @kurtmaile, I don't see anything in the REST API for transfer configs (https://cloud.google.com/bigquery-transfer/docs/reference/datatransfer/rest/v1/projects.locations.transferConfigs) that mentions range partitioning, though it's possible that it's supported via the params map, like in your potential config. That block is just a key/value map, so you could try putting those fields in there to see if it works. What happens if you try it?

[melinath]

If @danawillow's suggestion doesn't work, this could be something to look into adding.