Java v11 ⋅ Spring v2 ⋅ Gradle ⋅ BigQuery
-
Updated
Aug 18, 2023 - Java
Google BigQuery enables companies to handle large amounts of data without having to manage infrastructure. Google’s documentation describes it as a « serverless architecture (that) lets you use SQL queries to answer your organization's biggest questions with zero infrastructure management. BigQuery's scalable, distributed analysis engine lets you query terabytes in seconds and petabytes in minutes. » Its client libraries allow the use of widely known languages such as Python, Java, JavaScript, and Go. Federated queries are also supported, making it flexible to read data from external sources.
📖 A highly rated canonical book on it is « Google BigQuery: The Definitive Guide », a comprehensive reference.
Another enriching read on the subject is the inside story told in the article by the founding product manager of BigQuery celebrating its 10th anniversary.
Java v11 ⋅ Spring v2 ⋅ Gradle ⋅ BigQuery
The Apache Beam program which reads nginx access logs from Google Cloud Pub/Sub, parses them, and saves into BigQuery.
Yo dawg, I heard you like queries so we put some BigQuery in your query so you can query BigQuery from your query
Maven plugin for managing BigQuery datasets, tables and views
This project demonstrates how to read data from JDBC and write it to BigQuery with encryption using Google Tink and KMS. It also provides an example of how to customize the encryption by specifying the PII columns.
A POC on the data warehousing solution provided by Google Cloud
Java v17⋅ Spring v3 ⋅ Gradle ⋅ BigQuery
Nifi Big Query Custom Processor
Flink connector for BigQuery
Motor ETL - Extraccion BD a la nube de google
Adtech Logs processing Pipeline with Apache Beam, Cloud Dataflow, Java, Protocol Buffer. | Data Analysis with BigQuery
This video present a real world use case developed with Apache Beam Java and launched with the serverless Dataflow runner in Google Cloud Platform. The job read a Json file from Cloud Storage, applies some transformations and write the result to a BigQuery table.
Sample code for Apache Beam to perform ETL from a stream-processing service (Pub/Sub) to BigQuery using Dataflow as the runner
Released May 19, 2010