New sink: postgres

[jszwedko]

A note for the community

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Use Cases

Users want to write data to Postgres from Vector.

Breaking this off from #939

Attempted Solutions

No response

Proposal

No response

References

• New sink: MySQL #6556
• New timescale sink #939

Version

vector 0.26.0

[hhromic]

This is certainly an interesting kind of sink: databases.
I wonder if it would make more sense to implement an ODBC sink instead of specific database sinks?
This way, any database that provides ODBC drivers (there are many already) could interface with Vector out of the box. In addition, probably would be much less maintenance burden for Vector than maintaining multiple specific sinks.

[jszwedko]

Agreed, supporting something like ODBC would let us hit a lot of them at once. https://github.com/pacman82/odbc-api seems promising as a Rust wrapper.

[zamazan4ik]

AFAIK, rsyslog implements both approaches: ODBC based and database-specific based. The first one allows using with the same plugin for as many databases as possible without database-specific features. The second one uses database-specific features.

[spencergilbert]

I was also going to suggest a more general sink, at least to start with

[thomasdba]

any updates ?

[vnagendra]

Unless I am missing something what is needed for Postgres specifically is already in the current Cargo.toml (according to the PRs on this repo anyway)..

https://github.com/vectordotdev/vector/pull/18391/files

My suggestion would be to have something that addresses a specific set of (potentially limited) use cases. For me, having the ability to write to Db (a specific table) avoids having 1 more thing in the middle like Redis/Rabbit/etc.,

This is a fair thing to do for a limited number of things received (logs, metrics, messages whatever). When I say limited I am talking 100K+/day -- that is enough for a good set of use cases. Hopefully nobody receiving 100K/min is going to write to a db :)