Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: support custom input types / output types, custom conversion #18417

Closed
kardianos opened this issue Dec 22, 2016 · 7 comments
Closed
Assignees
Milestone

Comments

@kardianos
Copy link
Contributor

@kardianos kardianos commented Dec 22, 2016

This is a master tracking issues for various requests to support custom parameter types and custom scan types. This includes may geometry types, slices, structured fields. This issue also tracks issues related to conversion of one type into another type.

  • Expand types accepted in driver.TYPE #18415
  • possible inconsistent handling of types in convertAssign #18101
  • support time.Time and arrays #17834
  • support slices? #16235
  • allow drivers to support additional Value types #13567

I'd like to flush out a proposal here before go1.8 is released so it can be implemented smoothly in the go1.9 window. I would expect few public API additions. There might be a new interface or two in sql or in sql/driver.

@asaf
Copy link

@asaf asaf commented Jan 7, 2017

Hey folks,

There's a ref to rana/ora#145 about Oracle (and friends) Stored Procedures OUT params support,

Just for inspiration, here's an example how JDBC statement support these.

Whether statements support this or multi-value types result can also work,
Either way, this is a big issue in the enterprise world,

Thanks.

@kardianos
Copy link
Contributor Author

@kardianos kardianos commented Jan 9, 2017

@asaf I'd also like to enable OUT params. I would see it being implemented in the public database/sql API by adding Out bool to the NamedArg struct, then pass in a pointer to the value in the Value field. The part I would need to work out is how the driver API works and looks like. Though to be honest, that might be the only thing required as arguments are already passed through. We would almost certainly need to tweak query argument validation (which is another area I'd like to address for go1.9).

@tgulacsi
Copy link
Contributor

@tgulacsi tgulacsi commented Jan 9, 2017

If pointers of Out args are allowed to pass through convertAssign, that'd be an easy solution.

@tgulacsi
Copy link
Contributor

@tgulacsi tgulacsi commented Jan 9, 2017

Regarding #18415, allowing custom converters would be one option.

My main problem with the current situation is that the driver has an OCINum, that may or may not be represented as an int32 or int64 or float6r, but always can be a string.

If sql could provide the type what the user is Scanning into, then the driver could make a more informative decision about the conversion.

Now it uses float64 as the least wrong.

The same is with a LOB: if the user scans into an io.Reader, then the driver don't have to slurp all the content into memory.

@nullbio
Copy link

@nullbio nullbio commented Jan 9, 2017

I'm one of the authors of SQLBoiler (https://github.com/vattle/sqlboiler), we're currently using our own extended types packages to handle this, but it'd be very nice if it was part of the stdlib. What we currently use:

https://github.com/nullbio/null and https://github.com/vattle/sqlboiler/tree/master/types

This allows us to support all types as nullable (not only the ones currently in stdlib), hstore, 1 dimensional Postgres arrays and JSON. Would be very cool to see these added to the stdlib so we could drop support for these ugly dependencies we've had to craft :-).

These are the mappings we currently use if you care to know/if it's of any help or relevance:

https://github.com/vattle/sqlboiler/blob/master/bdb/drivers/mysql.go#L269
https://github.com/vattle/sqlboiler/blob/master/bdb/drivers/postgres.go#L309

Thanks.

@gopherbot
Copy link

@gopherbot gopherbot commented Mar 23, 2017

CL https://golang.org/cl/38533 mentions this issue.

gopherbot pushed a commit that referenced this issue May 18, 2017
Previously all arguments were passed through driver.IsValid.
This checked arguments against a few fundamental go types and
prevented others from being passed in as arguments.

The new interface driver.NamedValueChecker may be implemented
by both driver.Stmt and driver.Conn. This allows
this new interface to completely supersede the
driver.ColumnConverter interface as it can be used for
checking arguments known to a prepared statement and
arbitrary query arguments. The NamedValueChecker may be
skipped with driver.ErrSkip after all special cases are
exhausted to use the default argument converter.

In addition if driver.ErrRemoveArgument is returned
the argument will not be passed to the query at all,
useful for passing in driver specific per-query options.

Add a canonical Out argument wrapper to be passed
to OUTPUT parameters. This will unify checks that need to
be written in the NameValueChecker.

The statement number check is also moved to the argument
converter so the NamedValueChecker may remove arguments
passed to the query.

Fixes #13567
Fixes #18079
Updates #18417
Updates #17834
Updates #16235
Updates #13067
Updates #19797

Change-Id: I89088bd9cca4596a48bba37bfd20d987453ef237
Reviewed-on: https://go-review.googlesource.com/38533
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@kardianos
Copy link
Contributor Author

@kardianos kardianos commented May 30, 2017

I believe all of the above issues have been taken care of by allowing drivers to implement driver.NamedValueChecker . There is also a standard way to handle OUTPUT values in go1.9.

@kardianos kardianos closed this May 30, 2017
@golang golang locked and limited conversation to collaborators May 30, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants
You can’t perform that action at this time.