Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for schema v1 requests with more parameters #210

Closed
cole-miller opened this issue Oct 3, 2022 · 4 comments · Fixed by #243
Closed

Support for schema v1 requests with more parameters #210

cole-miller opened this issue Oct 3, 2022 · 4 comments · Fixed by #243
Assignees

Comments

@cole-miller
Copy link
Contributor

cole-miller commented Oct 3, 2022

I've implemented server-side support for versions of the EXEC, EXEC_SQL, QUERY, and QUERY_SQL requests that can include up to 2^32 - 1 statement parameters in canonical/dqlite#407. We'll want to implement client-side support for these in go-dqlite. The higher-level interface in the client package will need to take into account that older dqlite servers won't handle the new-style requests properly (they will try and presumably fail to parse them in the old format), perhaps with some kind of graceful fallback to the more limited old-style requests.

@cole-miller cole-miller self-assigned this Oct 3, 2022
@cole-miller
Copy link
Contributor Author

Assigning myself for now, but happy to defer if someone else wants to implement it (I expect to be more focused on roadmap stuff for the remainder of this cycle).

@cole-miller cole-miller changed the title Support for new-style requests with more parameters Support for schema v1 requests with more parameters Oct 3, 2022
@koisg
Copy link

koisg commented Apr 7, 2023

Hi @cole-miller, any plans to implement this in the near future?

Using dqlite with SQLBoiler ORM is difficult due to this issue. SQLBoiler is often building queries that rely on using multiple parameters. In some cases it might be difficult to rewrite a query in a way that avoids providing multiple parameters without breaking ORM patterns. For example, loading foreign key relationships with qm.Load is causing too many parameters error.

orm.Foo(qm.Load(orm.FooRels.FooBar)).All(ctx, db)

// SQL queries
SELECT "foo".* FROM "foo";
SELECT * FROM "bar" WHERE ("bar"."foo_id" IN (?, <300 parameters>, ?));

// error response
Error:
	failed to eager load FooBar: failed to eager load bar: too many parameters (301) max = 255

I tried increasing increasing MaxTupleParams const and removing safeguard in message.go, which resolved all issues I had, even though I haven't made any other changes to wire protocol.
What changes should be made to wire protocol to make sure it's working as intended? Is removing safeguard enough, or is it better to change number of params type to uint32 or are there more changes necessary?

@cole-miller
Copy link
Contributor Author

@koisg -- thanks for posting here and reminding me to implement this :) See #243.

@koisg
Copy link

koisg commented Apr 17, 2023

@cole-miller Thanks for addressing this quickly!

I've tried your pull request and it resolves the issue I was running into earlier. Do you plan to merge it soon?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants