Use sqlx for query generation

[mariusor]

We currently use hand baked queries for the models package, which look a bit nasty and complicated.

I'm sure we can replace at least some of them with sqlx generated ones.

[ghost]

maybe use boltdb with raft ?The current SQL schema is very basic.
also then its very easy to use graphql on top then and do various queries from the front end directly.

If you want some examples let me know.

EDIT:
I see your using golang models with direct access to the DB, since you dont need any graphql to support Activity Stream spec. You could then just use boltdb with raft.
Also for high scaling liftbridge is the perfect fit i feel and will remove a ton of code and get you higher perf and scaling.
https://github.com/liftbridge-io/liftbridge

liftbridge itself uses boltdb and raft to make everything persisted and redundant.
https://github.com/liftbridge-io/liftbridge/blob/master/server/raft.go

If you have any questions about liftbridge buzz me.

[mariusor]

Thanx, I'll do some research.

But I don't want to touch the db from the front-end (at least not for now, until I finalize the data structures) as I want flexibility to query every storage back-end (API, db, cache, etc - currently just API, db implemented) in the same way and get the same level of information.

[ghost]

you dont have to touch the db from the front end. It was silly of me to describe it that way. Graphql does not do that, but instead just allows the fronet end to describe the queries.

Anyway, i really think that LiftBridge will accelerate everything for you. Its basically a Acitivty Stream engine. You can still use all the code you ave now on top but it will solve allot of hard problems for you that you will hit.

[mariusor]

Thank you again.

I'm not familiar with any of the projects you mentioned and will probably take me a bit of time to actually understand how/where they can help. For now I'm trying to do baby steps in what I hope is a good direction.

If you can spare the time to show some concrete examples I'm sure it would help a lot, but you've given me some good references, and that's plenty too. :) Cheers.

[ghost]

liftbridge is like kafka.
it does pub sub and so you can recieve and push data to people easily. SO good for activity streams.
liftbridge is new and build on nats. nats can do 600 K transactions a second on a laptop. Its fast.
but because liftbridges is new there are only the examples in the repo. Have a look and try it.
Because its 100% golang its VERY easy.

if you get stuck buzz me again.

[mariusor]

Moved database interaction to db package and it uses sqlx for item retrieval. It's not fully fleshed yet, but it's workable.