Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

database/sql: common interface for query functions in sql.DB and sql.Tx #14468

Open
KilledKenny opened this issue Feb 22, 2016 · 15 comments
Open

database/sql: common interface for query functions in sql.DB and sql.Tx #14468

KilledKenny opened this issue Feb 22, 2016 · 15 comments
Assignees
Milestone

Comments

@KilledKenny
Copy link
Contributor

@KilledKenny KilledKenny commented Feb 22, 2016

Hi,

I'm proposing a interface that implements all functions in sql.DB and sql.Tx that takes a query string as one of its arguments. This could be useful when implementing "dumb" functions that retrieves information but in itself dose not need to be prepared however might be used to verify whether to commit or rollback.

The interface I'm proposing is: (The name can of course be changed)

type QueryAble interface {
    Exec(query string, args ...interface{}) (sql.Result, error)
    Prepare(query string) (*sql.Stmt, error)
    Query(query string, args ...interface{}) (*sql.Rows, error)
    QueryRow(query string, args ...interface{}) *sql.Row
}

This is an example function Get that can be used with or without transactions.

func Get (q Queryable)(int){
    var test int
    res , _ := q.Query("SELECT COUNT(*) FROM table;")
    res.Next()
    res.Scan(&test)
    res.Close()
    return test
}

func GetExample(db *sql.DB){

    //When you just want to retrieve the information, no need for a transaction
    fmt.Printf("Current len of Table %d\n", Get(db))

    //Make a change
    tx, _ :=db.Begin()
    // Add data to table
    if  Get(tx) > 2 {
        fmt.Printf("Table to large: %d robacking to: %d \n", Get(tx) , Get(db))
        tx.Rollback()
    } else {
        fmt.Printf("Table update new len %d\n", Get(tx))
        tx.Commit()
    }
}
@ianlancetaylor ianlancetaylor changed the title database/sql Common interface for query functions in sql.DB and sql.Tx database/sql: Common interface for query functions in sql.DB and sql.Tx Feb 22, 2016
@ianlancetaylor ianlancetaylor changed the title database/sql: Common interface for query functions in sql.DB and sql.Tx database/sql: common interface for query functions in sql.DB and sql.Tx Feb 22, 2016
@ianlancetaylor ianlancetaylor added this to the Unplanned milestone Feb 22, 2016
@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 22, 2016

I'm sorry, I don't entirely understand this proposal. Are you suggesting that we should add the type QueryAble to database/sql? How would that make the package easier to use than it is today?

@KilledKenny

This comment has been minimized.

Copy link
Contributor Author

@KilledKenny KilledKenny commented Feb 22, 2016

Yes, I want to add QueryAble to database/sql

The reason is that: I want to be able to create functions that execute SQL query's and retrieve data regardless of if its inside a prepared statement (and preferably only use prepared statements when i really need to). This is not possible without a interface.

To provide a transaction and non transaction solution today you have to do something like this:

func SqlMathWrapper(db *sql.DB , a, b int)(int, error){
    tx, _ := db.Begin()
    defer tx.Commit()
    return SqlMath(tx, a,b)
}

func SqlMath(tx *sql.Tx, a, b int)(num int, err error){
    err = tx.QueryRow("SELECT ? + ?", a, b).Scan(&num)
    return
}

(This dose always use transaction)

Using the interface the code would look like this

func SqlMath(qa *sql.QueryAble, a, b int)(num int, err error){
    err = qa.QueryRow("SELECT ? + ?", a, b).Scan(&num)
    return
}

(This would not use prepared statements if called with a sql.DB object)

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Feb 22, 2016

Just to be clear, you could define the interface yourself, right?

@KilledKenny

This comment has been minimized.

Copy link
Contributor Author

@KilledKenny KilledKenny commented Feb 22, 2016

@ianlancetaylor Yes, and that's what I will do until its implemented.

@kardianos

This comment has been minimized.

Copy link
Contributor

@kardianos kardianos commented Oct 8, 2016

Another discission at #14468 .

@kardianos

This comment has been minimized.

Copy link
Contributor

@kardianos kardianos commented Nov 28, 2016

The API for rollback / savepoint may also factor into this conversation: #7898.

@vwochnik

This comment has been minimized.

Copy link

@vwochnik vwochnik commented May 9, 2017

I am strongly in favor of the Queryable interface as it's proposed here. At the company I am working for I am running into the same use case where I have a few CRUD functions which don't need to care if it's a sql.DB or sql.Tx. In fact, it's counter intuitive. I want to use these functions both with and without transactions. The responsibility of handling the transactions is outside of the scope of the functions.

@kardianos

This comment has been minimized.

Copy link
Contributor

@kardianos kardianos commented May 9, 2017

@vwochnik I'm not convinced such an interface needs to live in the std lib.
Here is one implementation: https://godoc.org/github.com/golang-sql/sqlexp#Querier
You could also define your own.

@kPshi

This comment has been minimized.

Copy link

@kPshi kPshi commented May 10, 2017

Absolutely common issue it seems and I have the same case here. Of course we can all write an interface definition on our own but that would be done again and again causing much more work in the end (when summing up each individual time spent or working around that).
Libraries are used to ease up things and to not make everyone repeat the very same work. So the question in my opinion is not whether one is able to write that but the question is if that would make things easier and for how many people.

@kardianos

This comment has been minimized.

Copy link
Contributor

@kardianos kardianos commented May 10, 2017

Great @kPshi . Start using the interface defined in sqlexp and spread the word. Then when the tree opens in 3 months we can see how widely it is used.

@posener

This comment has been minimized.

Copy link

@posener posener commented Feb 24, 2018

How can one use an http.RoundTripper-like interface for SQL queries? The http library has really nice behaviors/abstractions/profiling hooks around the http.Client, which we miss in the sql.DB client.

@kardianos

This comment has been minimized.

Copy link
Contributor

@kardianos kardianos commented Feb 24, 2018

@posener

This comment has been minimized.

Copy link

@posener posener commented Feb 24, 2018

Not at this level. Some drivers a not network based, others are cgo based.
Ask the driver for a connector with that capacity.

Why? They all implement the ExecContext, QueryContext, and so... function for example, wouldn't it be nice to have an option to "wrap" calls for those function, and add instrumentation, such as logging, timing, and so?
Why should it be a driver-specific functionality?

Some use cases I thought of:

  • If I'm using an ORM, like gorm, I am "stuck" with the logging gorm had defined.
  • If I want to use open tracing, or server timing, I have to run function before and after each call for a database exec/query and can't wrap in a general way all the calls to the database.

I could have a struct implementing the non-existent interface, that wrap a given database connection:

type SQLLogger struct {
    *sql.DB
}

func (s *SQLLogger) ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error) {
    log.Printf("whatever")
    defer log.Printf("finished!")
    return s.DB.ExecContext(ctx, query, args...)
}

The problem is that since this interface does not exists in the standard library, most libraries expect an *sql.DB and can not accept the new instrument struct that implements the query/exec function.

@romeovs

This comment has been minimized.

Copy link

@romeovs romeovs commented Aug 23, 2018

I agree.

Libraries are used to ease up things and to not make everyone repeat the very same work. So the question in my opinion is not whether one is able to write that but the question is if that would make things easier and for how many people.

This is what makes Go so powerful IMHO: a well-defined set of shared libraries that can work together seamlessly, because most high-level concepts are expertly expressed in the STL.

@Vitucho

This comment has been minimized.

Copy link

@Vitucho Vitucho commented Dec 30, 2019

I have the same issue of the author, so 👍

Why don't you define a new interface like DbTx honoring tha pattern invented for the interface like
TB. Leaving aside functional differences the "scenario" is quite the similar.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants
You can’t perform that action at this time.