Compress text API responses before leaving data centre

[DevOfManyThings]

Feature request

Is your feature request related to a problem? Please describe.

Currently Supabase compresses text payloads at the CDN (confirmed in email with support): https://supabase.com/blog/2021/12/03/launch-week-three-friday-five-more-things

This means the egress data sent from Supabase's servers which manage the pooled connections to the database send out the raw payloads uncompressed, which uses significantly more data than if compression were to be moved to those servers rather than handled by the CDN.

https://paulcalvano.com/2018-07-25-brotli-compression-how-much-will-it-reduce-your-content/

Describe the solution you'd like

Supabase to move the point of compression to the servers which expose the REST API for text base responses.

This compression should reflect in the egress data usage.

As the data is already being sent to the client compressed, this change could be handled by Supabase transparently without requiring updates by end users, and cause a significant drop in their egress data bill.

Describe alternatives you've considered

N/A

Additional context

This won't help applications which connect directly to the database or storage API, but would be very beneficial to users of the REST api.

[steve-chavez]

@DevOfManyThings Makes sense. Also doing compression on the database is possible by using pgbrotli or pgsql-gzip. So really this could benefit all postgres clients.

If we had the above mentioned extensions this could already be done in SQL functions(RPC). Also doing it automatically for tables/views should be possible later.

[DevOfManyThings]

@steve-chavez good idea, Supabase would need to make it available on their extensions page as it's not currently available at the moment.
I'd hope other than that it should be a pretty seamless transition to enable it but I assume they'd do some testing to make sure it doesn't break existing functionality.

[DevOfManyThings]

@steve-chavez
As I understand it PostgREST runs on a different host to the database instance (whilst still within aws datacentre).
So it goes: database -> postgREST -> exits AWS datacentre -> CDN -> supabase client, I've not went into specifics but I imagine using those plugins won't actually help users of postgREST benefit from compression.

Would this be better split into two issues, one to request those plugins on supabase for those who connect directly via TCP, and another for those that connect via postgREST (which I believe includes graphQL users).

[steve-chavez]

As I understand it PostgREST runs on a different host to the database instance

@DevOfManyThings This is true right now but we're planning to move postgREST to the db instance in the near future.

[DevOfManyThings]

@MildTomato can you provide any details why you changed this to a discussion?
Considering another employee engaged with this and seemed to think it was worth some merit I thought it was just a low priority

[steve-chavez]

Probably because is not actionable right now. But I'll come back to this one, I might have something interesting regarding compression for the next postgREST release.

[rpock]

Now that we're between us, what can you tell us about what's new at PostgREST @steve-chavez 😏🤫?

[steve-chavez]

PostgREST already has custom media type handlers, which makes it possible to use some compressed types: https://postgrest.org/en/v12/references/api/media_type_handlers.html#handler-function

But this capability delegates the work to postgres extensions, so Supabase would need to offer pgsql-gzip for example. Which is not available.

[rpock]

A brotli postgres extension would be very nice 🙌 or at least a pgsql_gzip to use it in postgres functions to return compressed data to the client. In my case this would reduce the amount of data up to 90% for arrays of datasets.

Brotli was mentioned at the end of 2021 but for other use cases inside Supabase
https://supabase.com/blog/launch-week-three-friday-five-more-things