Ability to implement on GCP with cloud storage interoperability

[rajatha-ravish]

No description provided.

[darrenhaken]

Any ETA on this? we're databricks customers who'd love to use this on GCS/GCP

[rtyler]

@darrenhaken this will unfortunately sit in the issues list until somebody steps up to add GCP support similar to #136 which was recently contributed for Azure support

[mightyshazam]

@rtyler @darrenhaken With #136 merged, it may be as simple as adding gcs to the features. Then, according to the object store code, users just need to change there uri to gs:// instead of an s3 or azure prefix. It will require testing, but it may be an easy lift.

[rtyler]

One of the things that I recently learned is that GCS supports S3 compatibility through their "Interoperability" feature, so it would technically be possible to just configure kafka-delta-ingest using the S3-compatible APIs for GCS.

[geoHeil]

Is a simple repoint of the S3 URI using AWS_ENDPOINT_URL enough for this change?