Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: common interface for query functions in sql.DB and sql.Tx #14468

Open
KilledKenny opened this issue Feb 22, 2016 · 16 comments
Open

database/sql: common interface for query functions in sql.DB and sql.Tx #14468

KilledKenny opened this issue Feb 22, 2016 · 16 comments
Assignees
Milestone

Comments

@KilledKenny
Copy link
Contributor

@KilledKenny KilledKenny commented Feb 22, 2016

Hi,

I'm proposing a interface that implements all functions in sql.DB and sql.Tx that takes a query string as one of its arguments. This could be useful when implementing "dumb" functions that retrieves information but in itself dose not need to be prepared however might be used to verify whether to commit or rollback.

The interface I'm proposing is: (The name can of course be changed)

type QueryAble interface {
    Exec(query string, args ...interface{}) (sql.Result, error)
    Prepare(query string) (*sql.Stmt, error)
    Query(query string, args ...interface{}) (*sql.Rows, error)
    QueryRow(query string, args ...interface{}) *sql.Row
}

This is an example function Get that can be used with or without transactions.

func Get (q Queryable)(int){
    var test int
    res , _ := q.Query("SELECT COUNT(*) FROM table;")
    res.Next()
    res.Scan(&test)
    res.Close()
    return test
}

func GetExample(db *sql.DB){

    //When you just want to retrieve the information, no need for a transaction
    fmt.Printf("Current len of Table %d\n", Get(db))

    //Make a change
    tx, _ :=db.Begin()
    // Add data to table
    if  Get(tx) > 2 {
        fmt.Printf("Table to large: %d robacking to: %d \n", Get(tx) , Get(db))
        tx.Rollback()
    } else {
        fmt.Printf("Table update new len %d\n", Get(tx))
        tx.Commit()
    }
}
@ianlancetaylor ianlancetaylor changed the title database/sql Common interface for query functions in sql.DB and sql.Tx database/sql: Common interface for query functions in sql.DB and sql.Tx Feb 22, 2016
@ianlancetaylor ianlancetaylor changed the title database/sql: Common interface for query functions in sql.DB and sql.Tx database/sql: common interface for query functions in sql.DB and sql.Tx Feb 22, 2016
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Feb 22, 2016
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 22, 2016

I'm sorry, I don't entirely understand this proposal. Are you suggesting that we should add the type QueryAble to database/sql? How would that make the package easier to use than it is today?

@KilledKenny
Copy link
Contributor Author

@KilledKenny KilledKenny commented Feb 22, 2016

Yes, I want to add QueryAble to database/sql

The reason is that: I want to be able to create functions that execute SQL query's and retrieve data regardless of if its inside a prepared statement (and preferably only use prepared statements when i really need to). This is not possible without a interface.

To provide a transaction and non transaction solution today you have to do something like this:

func SqlMathWrapper(db *sql.DB , a, b int)(int, error){
    tx, _ := db.Begin()
    defer tx.Commit()
    return SqlMath(tx, a,b)
}

func SqlMath(tx *sql.Tx, a, b int)(num int, err error){
    err = tx.QueryRow("SELECT ? + ?", a, b).Scan(&num)
    return
}

(This dose always use transaction)

Using the interface the code would look like this

func SqlMath(qa *sql.QueryAble, a, b int)(num int, err error){
    err = qa.QueryRow("SELECT ? + ?", a, b).Scan(&num)
    return
}

(This would not use prepared statements if called with a sql.DB object)

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 22, 2016

Just to be clear, you could define the interface yourself, right?

@KilledKenny
Copy link
Contributor Author

@KilledKenny KilledKenny commented Feb 22, 2016

@ianlancetaylor Yes, and that's what I will do until its implemented.

@kardianos
Copy link
Contributor

@kardianos kardianos commented Oct 8, 2016

Another discission at #14468 .

@kardianos
Copy link
Contributor

@kardianos kardianos commented Nov 28, 2016

The API for rollback / savepoint may also factor into this conversation: #7898.

@vwochnik
Copy link

@vwochnik vwochnik commented May 9, 2017

I am strongly in favor of the Queryable interface as it's proposed here. At the company I am working for I am running into the same use case where I have a few CRUD functions which don't need to care if it's a sql.DB or sql.Tx. In fact, it's counter intuitive. I want to use these functions both with and without transactions. The responsibility of handling the transactions is outside of the scope of the functions.

@kardianos
Copy link
Contributor

@kardianos kardianos commented May 9, 2017

@vwochnik I'm not convinced such an interface needs to live in the std lib.
Here is one implementation: https://godoc.org/github.com/golang-sql/sqlexp#Querier
You could also define your own.

@kPshi
Copy link

@kPshi kPshi commented May 10, 2017

Absolutely common issue it seems and I have the same case here. Of course we can all write an interface definition on our own but that would be done again and again causing much more work in the end (when summing up each individual time spent or working around that).
Libraries are used to ease up things and to not make everyone repeat the very same work. So the question in my opinion is not whether one is able to write that but the question is if that would make things easier and for how many people.

@kardianos
Copy link
Contributor

@kardianos kardianos commented May 10, 2017

Great @kPshi . Start using the interface defined in sqlexp and spread the word. Then when the tree opens in 3 months we can see how widely it is used.

@posener
Copy link

@posener posener commented Feb 24, 2018

How can one use an http.RoundTripper-like interface for SQL queries? The http library has really nice behaviors/abstractions/profiling hooks around the http.Client, which we miss in the sql.DB client.

@kardianos
Copy link
Contributor

@kardianos kardianos commented Feb 24, 2018

@posener
Copy link

@posener posener commented Feb 24, 2018

Not at this level. Some drivers a not network based, others are cgo based.
Ask the driver for a connector with that capacity.

Why? They all implement the ExecContext, QueryContext, and so... function for example, wouldn't it be nice to have an option to "wrap" calls for those function, and add instrumentation, such as logging, timing, and so?
Why should it be a driver-specific functionality?

Some use cases I thought of:

  • If I'm using an ORM, like gorm, I am "stuck" with the logging gorm had defined.
  • If I want to use open tracing, or server timing, I have to run function before and after each call for a database exec/query and can't wrap in a general way all the calls to the database.

I could have a struct implementing the non-existent interface, that wrap a given database connection:

type SQLLogger struct {
    *sql.DB
}

func (s *SQLLogger) ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error) {
    log.Printf("whatever")
    defer log.Printf("finished!")
    return s.DB.ExecContext(ctx, query, args...)
}

The problem is that since this interface does not exists in the standard library, most libraries expect an *sql.DB and can not accept the new instrument struct that implements the query/exec function.

@romeovs
Copy link

@romeovs romeovs commented Aug 23, 2018

I agree.

Libraries are used to ease up things and to not make everyone repeat the very same work. So the question in my opinion is not whether one is able to write that but the question is if that would make things easier and for how many people.

This is what makes Go so powerful IMHO: a well-defined set of shared libraries that can work together seamlessly, because most high-level concepts are expertly expressed in the STL.

@vituchon
Copy link

@vituchon vituchon commented Dec 30, 2019

I have the same issue of the author, so 👍

Why don't you define a new interface like DbTx honoring tha pattern invented for the interface like
TB. Leaving aside functional differences the "scenario" is quite the similar.

@arianitu
Copy link

@arianitu arianitu commented Mar 14, 2020

I'm a 👍 on this one, it would be nice to be able to have a shared interface that you can pass a Txt or a normal db.

We ended up creating our own interface which isn't too bad, it just may spook some people coming to the codebase that see db.Querier instead of something like sql.Querier

I think Querier and Queryable are actually tricky to type so we just went with db.Conn

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.