Publish "etl-core" as separate library on crates.io

[marcomq]

Hi!
I just found your etl project https://github.com/supabase/etl/. Opening a new issue forwarded me to this repository.
It is kind of great to see that someone started to create WAL and data replication code for postgres with rust! This is something that was missing yet.

I'm building an mq / stream library in rust - https://github.com/marcomq/mq-bridge/. I currently still read my postgres data with sqlx in polling mode. I already made some changes to my dev branch to improve reading from postgres data via WAL/CDC and forward it to some other stream, queue or database, but I couldn't find any reliable solution yet.

Anyway - it would be great to have a library for this use case, so I would somehow be able to read postgres WAL in a reliable fast way and don't need to use unreliable slow polling with sqlx. It would be great if you could offer this part as separate library.

Maybe you could even think about using mq-bridge in etl as additional feature. It might be a solution to forward or publish your data to a message queue or stream. There is already support for kafka, nats, rabbitmq, zeromq, redis streams, AWS sqs etc... You could easily add multiple streaming and queue protocols just by using a single API. It would even easily add sqlx (mariadb, mysql, sqlite) and mongodb support. It even has an npm lib.