Local sql query able sink

[gedw99]

Genji is a sql store based on badger dB. Been getting very high perf when using it on projects because it’s embedded and because it’s badgerdb and not SQLite.

would you consider adding it as a sink store ?

https://github.com/genjidb/genji

genji has no HA aspects but I think it’s possible in the future using raft badger / pebble implementations out there

I am considering adding benthos to allow scriptable transformations also.

https://github.com/Jeffail/benthos

[kelindar]

@gedw99 I just realised that I forgot to reply to this. Genji could potentially be added, but it would create a duplicate of the data. Also not sure if it's going to fit well with the columnar design of talaria.

Benthos would be very good sink candidate indeed, PRs are welcome as we've started refactoring some of the streaming stuff here. Also still need to add kafka/avro support :-)

[gedw99]

its cool.

genji is pretty flexible, since its just badgerdb under it.
But, looking at how it would work as a sin with Talaria, I cant seem to find in the current code how the columns are structured and how a query works with it.

Regarding Benthos. sure. benthos does not need kafka / avro btw.