Skip to content

out_bigquery: add skipInvalidRows and ignoreUnknownValues params #623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 11, 2021
Merged

out_bigquery: add skipInvalidRows and ignoreUnknownValues params #623

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions pipeline/outputs/bigquery.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,10 @@ Fluent Bit BigQuery output plugin uses a JSON credentials file for authenticatio
| project\_id | The project id containing the BigQuery dataset to stream into. | The value of the `project_id` in the credentials file |
| dataset\_id | The dataset id of the BigQuery dataset to write into. This dataset must exist in your project. | |
| table\_id | The table id of the BigQuery table to write into. This table must exist in the specified dataset and the schema must match the output. | |
| skip_invalid_rows | Insert all valid rows of a request, even if invalid rows exist. The default value is false, which causes the entire request to fail if any invalid rows exist. | Off |
| ignore_unknown_values | Accept rows that contain values that do not match the schema. The unknown values are ignored. Default is false, which treats unknown values as errors. | Off |

See Google's [official documentation](https://cloud.google.com/bigquery/docs/reference/rest/v2/tabledata/insertAll) for further details.

## Configuration File

Expand Down