Releases: dforsber/glue-table-cache
Releases · dforsber/glue-table-cache
Release list
v0.9.21
v0.9.19
v0.9.18
v0.9.17
🚀 New optional proxyAddress parameter for enabling local S3 Object caching
Add new optional parameter proxyAddress (e.g. http://locahost:3203/). When present the S3 URLs will be replaced with URLs pointing to the proxy address.
With proxyAddress: "http://localhost:3203" set, DuckDB fetches the S3 Objects from e.g. http://localhost:3203/BUCKET/PREFIX address instead of s3://BUCKET/PREFIX.
Discussion about proxy Implementations
The proxy should be transparent proxy towards S3.
An example implementation could for example download S3 Objects to local /tmp folder whenever requested. This reduces S3 API calls, but more importantly makes the DuckDB's Glue Table access like reading local files instead of running lots of round trip requests, which increases performance tenfolds.