db.Query breaks the application over ab benchmark #14
Comments
|
Could you please try to use mutex.Lock/Unlock between querying? |
|
I'm learning Go and not started with concurrency, so do not know how to use mutex. I've tried: And also putting the Unlock after the rows.Next loop, but without success. I thought that, although unlikely, could be a problem of "database/sql", so I used the same example with the PostgreSQL driver ( bmizerany/pq ) and a similar DB, but all ab test passed flawless. As I understand, each http request runs in its own goroutine, but can share the database connection. When I run requests manually there are no concurrency, as the process is fast enough to have ended when the next request arrive, and therefore do not share the connection. But a program like ab is able to send many more requests per second, and can do so concurrently, so it is probable to run a new db.Query without having received the previous query data. pq has had a similar problem, but only with transactions, maybe this can throw some light: https://groups.google.com/forum/#!topic/golang-nuts/94NKlXjbN1g |
|
Ok, I'll look into it. |
|
I didn't get error with following code. package main
import (
"fmt"
"net/http"
"database/sql"
_ "github.com/mattn/go-adodb"
"sync"
)
var mutex sync.Mutex
func rootHandler(w http.ResponseWriter, r *http.Request) {
mutex.Lock()
defer mutex.Unlock()
db, err := sql.Open("adodb", "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=./example.mdb;")
if err != nil {
fmt.Println(err)
return
}
defer db.Close()
rows, err := db.Query("select id, name from languages")
if err != nil {
fmt.Println(err)
return
}
defer rows.Close()
for rows.Next() {
var id string
var name string
rows.Scan(&id, &name)
fmt.Println(id, name)
}
}
func main() {
http.HandleFunc("/", rootHandler)
fmt.Println("About to listen on 8081")
err := http.ListenAndServe(":8081", nil)
if err != nil {
fmt.Println("ListenAndServe: ",err)
return
}
} |
|
Yes, your code works, the key is to declare the mutex variable globally and make mutex.Lock() at the beginning of the handler. Thank you. To not set a connection on each request, and get higher requests per second, I made a test declaring a global variable db * sql.DB and calling sql.Open in main, but then, after 500-600 requests throws error "db.Query error: An exception occurred". See code: http://play.golang.org/p/hB0XHYi7Kc |
|
Looks good |
|
I have the same exception, but use no currency at all. The crash is repetitive, and depending on the order I execute the queries. I.e.: if I execute query1 before and query2 after, it crashes. If I execute the same exact query2 before and query1 after, it works. The crash occours only if, after db.Query(...), I read the data using rows.Next(). If I just execute the db.Query(...), it does not crash. Can you help me diagnose the issue? Stack is: |
|
My problem is indeed a duplicate #18, which is in turn a duplicate of golang/go#9457: adding |
|
At the first, I don't still reproduce this. So I can't put the workaround in README.md . |
|
Anyway, it seems a way to save my (or your's) time. updated README. Thanks. |
I have divided the mdb.go example in two apps:
When I test the mdb2 app manually with the browser, curl, etc it works fine, but when y put them on load, for example:
ab -n 100 -c 10 -k http://app.url/
ab exits with "...apr_socket_recv: Connection reset by peer (54)", and the app crashes with:
I have test it wit wrk with similar errors:
wrk -t2 -c10 -d5s http://app.url
Thanks.
The text was updated successfully, but these errors were encountered: