Docker Spark

docker pull ghcr.io/hienduyph/spark:latest

Useful Jars

• https://iceberg.apache.org/

Google CLoud BigQuery:

• gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-CONNECTOR_VERSION.jar
• https://github.com/GoogleCloudDataproc/spark-bigquery-connector/releases

GCS:

• https://storage.googleapis.com/hadoop-lib/gcs/gcs-connector-hadoop3-latest.jar
• gs://hadoop-lib/gcs/gcs-connector-hadoop3-latest.jar