Working on API updates for the Store & Models

[mvoorberg]

...likely needs more work, but it's coming along.

On my wishlist:

• Select column lists
• Pagination w/ skip & limit
• Ordering
• Partial update
• Update to NULL
• Select where ISNULL
• ...then IN list support

[aakash-rajur]

w.r.t your store design

insert

1. existing Insert is variadic and hence supports both InsertOne and InsertBulk.
2. if your use case is to count the no of inserts, you get inserted records as output of Insert on which you can do length check while also keeping the whole mechanism functional

update

same as insert hence don't see how UpdateOne helps?

find

1. Find and Update are the most used operations within a transaction in a mutation
2. if you're operating on multiple rows simultaneously, FindMany and Update (since it's variadic), fits the use case
3. i don't see how FindOne and FindSingle use cases are not already satisfied with existing design?
4. FindExactlyOne is usually taken care by unique constraint in db so duplicating this logic in application seems redundant!
5. FindByPk is also satisfied with Find or FindMany

   id := 1
   name := "John Doe"
   
   // find actor by primary key using `Find`
   actor, err := store.Find(&model.Actor{Id: &id})
   
   // find same actor by name using `Find`
   actor, err := store.Find(&model.Actor{Name: &name});
   
   // find same actor by name using `Find` and vector index search
   actor, err := store.Find(&model.Actor{NameSearch: &name});

delete

1. i understand DeleteByPk use case but what i've found in retrospect that most existing ORM designs use all columns provided to do a delete to be extra sure that they're deleting the right row
2. having visited the code again, i feel an improvement over Delete would be to return the deleted row count and accept partial columns to delete many rows at once! (which i'll add shortly!)

count and pagination

1. there are multiple ways of achieving this, one as you've been suggesting is to have a separate count query which in all fairness is more readable
2. a more efficient way of achieving this is to use window functions supported by both postgresql and mysql, this avoid an extra io or network request to db and fetches the result in single query.
3. an even more efficient way would be to use id of last item in a page to fetch next page, this avoids the offset sort performance penalty when your tables are huge
4. the design you suggest only supports the first use case while a lot of people opt-in for the 2nd approach to begin with and move to 3rd when data grows beyond a threshold

custom

1. in case none of the generated code satisfy your case, you can write custom queries as specified in the example directory and the tool will generate models for input and output
2. for the need to have further control, you can directly interact with sqlx as this tool only generates code and is not directly involved!

now if you still feel comfortable with the design you're suggesting i highly recommend you to checkout sqlboiler, it has all the traditional use case you're looking for and lot more than you can imagine

highly appreciate your previous PR for my obvious misses!

[mvoorberg]

On Sun, Feb 11, 2024, 12:08 a.m. Aakash Rajur ***@***.***> wrote: w.r.t your store design <https://github.com/aakash-rajur/sqlxgen/pull/2/files#diff-c6802b187b78c42aa0d89bb284aa50fcfe21d7551693357a74f74ce6073a3315> insert 1. existing Insert is variadic and hence supports both InsertOne and InsertBulk.

Yes, and I love it that way. My use case is to insert as many as I can in as few trips to the dB as possible. E.g.100,000s What is lost this way is the inserted pks and reselected rows.

1. if your use case is to count the no of inserts, you get inserted records as output of Insert on which you can do length check while also keeping the whole mechanism functional update same as insert hence don't see how UpdateOne helps?

UpdateOne tells other developers (and future me) my intention, as well as being more explicit. Pulling back only the single row is just convenient. If my update were to match multiples, it should be a hard failure.

find 1. Find and Update are the most used operations within a transaction in a mutation 2. if you're operating on multiple rows simultaneously, FindMany and Update (since it's variadic), fits the use case 3. i don't see how FindOne and FindSingle use cases are not already satisfied with existing design?

findSingle is (lowercase f) internal, used by findOne and findByPk. Again, both are more explicit.

1. FindByPk is also satisfied with Find or FindMany id := 1name := "John Doe" // find actor by primary key using `Find`actor, err := store.Find(&model.Actor{Id: &id}) // find same actor by name using `Find`actor, err := store.Find(&model.Actor{Name: &name}); // find same actor by name using `Find` and vector index searchactor, err := store.Find(&model.Actor{NameSearch: &name}); delete 1. i understand DeleteByPk use case but what i've found in retrospect that most existing ORM designs use all columns provided to do a delete to be extra sure that they're deleting the right row

More 'extra sure' than the pk doesn't make sense. More explicit is good. Keep in mind my object may be created 3 layers away from where the delete() is called!

1. having visited the code again, i feel an improvement over Delete would be to return the deleted row count and accept partial columns to delete many rows at once! (which i'll add shortly!)

While this will be useful, it would scare me to use as wrong parameters could be devastating.

count and pagination 1. there are multiple ways of achieving this, one as you've been suggesting is to have a separate count query which in all fairness is more readable 2. a more efficient way of achieving this is to use window functions supported by both postgresql and mysql, this avoid an extra io or network request to db and fetches the result in single query.

Skip and limit are well known paradigms used in many places and even ui widgets. Redesigning that seems like making it more difficult to use.

1. an even more efficient way would be to use id of last item in a page to fetch next page, this avoids the offset sort performance penalty when your tables are huge 2. the design you suggest only supports the first use case while a lot of people opt-in for the 2nd approach to begin with and move to 3rd when data grows beyond a threshold custom 1. in case none of the generated code satisfy your case, you can write custom queries as specified in the example directory and the tool will generate models for input and output 2. for the need to have further control, you can directly interact with sqlx as this tool only generates code and is not directly involved!

Custom sql is always an option, I will be using that!

now if you still feel comfortable with the design you're suggesting i highly recommend you to checkout sqlboiler <https://github.com/volatiletech/sqlboiler>, it has all the traditional use case you're looking for and lot more than you can imagine

Sqlboiler looks cool, but I like the simplicity of sqlxgen - even if I'm proposing additions!

highly appreciate your previous PR for my obvious misses!

Happy to contribute, I learned some cool Go techniques going through it!

…

— Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6EM5Z7YPT2EF35MZSSEYLYTB4BLAVCNFSM6AAAAABDDAANYCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZXGQ3DMMZTGE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[aakash-rajur]

ok, can we start with an issue that catalogues all the use case you think are not covered or you think is less accessible?
for ex, bulk insert in a single round trip seems to a very valid use case.

we're jumping to solutions before understanding problem(s), what do you think?

[mvoorberg]

Fair enough. *FindOne* and *FindByPk*, have a look at this orm for MongoDb https://mongoosejs.com/docs/queries.html They make the distinction between: *findOne* and *findById* (findByPk in our case) Finds a single document by its _id field. findById(id) is almost* equivalent to findOne({ _id: id }). If you want to query by a document's _id, use findById() instead of findOne(). *FindFirst* allows the possibility that more than one record matches your criteria - when you don't require a unique guarantee. See this method from C# linq: https://learn.microsoft.com/en-us/dotnet/api/system.linq.enumerable.firstordefault?view=net-8.0#system-linq-enumerable-firstordefault-1(system-collections-generic-ienumerable((-0))-system-func((-0-system-boolean))-0) *Find* - already exists, matching on all fields provided and (quietly) limiting to a single row. *FindMany* - already exists, matching on all fields provided. *Count *- match all provided fields, returns only the number or an error *DeleteByPk* - Use the PK for fast access and guaranteed single record delete. *DeleteOne* - Used when a table has a UQ but no PK, or using another unique set of fields. *Delete* (match all fields provided) - Used when you want to delete multiples, or a complete record that has not changed since you fetched it. *Update* - exists, update one row using the Pk There are other Update use cases that it should support - but I haven't fully wrapped my head around them yet. E.g. Updating multiple rows to match a provided set of attributes - M

…

On Mon, Feb 12, 2024, 1:20 a.m. Aakash Rajur ***@***.***> wrote: ok, can we start with an issue that catalogues all the use case you think are not covered or you think is less accessible? for ex, bulk insert in a single round trip seems to a very valid use case. — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB6EM5YIL22CYILFAWVRMRTYTHNGXAVCNFSM6AAAAABDDAANYCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMZYGI4TKOJQGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[aakash-rajur]

can you create an issue? we can move the discussion there, discuss strategies and solutions. Then we can create a PR addressing this issue

i don't mind creating it but I want it to have you as the author and your idea and extension