-
-
Notifications
You must be signed in to change notification settings - Fork 36
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
db.Query breaks the application over ab benchmark #14
Comments
Could you please try to use mutex.Lock/Unlock between querying? |
I'm learning Go and not started with concurrency, so do not know how to use mutex. I've tried:
And also putting the Unlock after the rows.Next loop, but without success. I thought that, although unlikely, could be a problem of "database/sql", so I used the same example with the PostgreSQL driver ( bmizerany/pq ) and a similar DB, but all ab test passed flawless. As I understand, each http request runs in its own goroutine, but can share the database connection. When I run requests manually there are no concurrency, as the process is fast enough to have ended when the next request arrive, and therefore do not share the connection. But a program like ab is able to send many more requests per second, and can do so concurrently, so it is probable to run a new db.Query without having received the previous query data. pq has had a similar problem, but only with transactions, maybe this can throw some light: https://groups.google.com/forum/#!topic/golang-nuts/94NKlXjbN1g |
Ok, I'll look into it. |
I didn't get error with following code. package main
import (
"fmt"
"net/http"
"database/sql"
_ "github.com/mattn/go-adodb"
"sync"
)
var mutex sync.Mutex
func rootHandler(w http.ResponseWriter, r *http.Request) {
mutex.Lock()
defer mutex.Unlock()
db, err := sql.Open("adodb", "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=./example.mdb;")
if err != nil {
fmt.Println(err)
return
}
defer db.Close()
rows, err := db.Query("select id, name from languages")
if err != nil {
fmt.Println(err)
return
}
defer rows.Close()
for rows.Next() {
var id string
var name string
rows.Scan(&id, &name)
fmt.Println(id, name)
}
}
func main() {
http.HandleFunc("/", rootHandler)
fmt.Println("About to listen on 8081")
err := http.ListenAndServe(":8081", nil)
if err != nil {
fmt.Println("ListenAndServe: ",err)
return
}
} |
Yes, your code works, the key is to declare the mutex variable globally and make mutex.Lock() at the beginning of the handler. Thank you. To not set a connection on each request, and get higher requests per second, I made a test declaring a global variable db * sql.DB and calling sql.Open in main, but then, after 500-600 requests throws error "db.Query error: An exception occurred". See code: http://play.golang.org/p/hB0XHYi7Kc |
Looks good |
I have the same exception, but use no currency at all. The crash is repetitive, and depending on the order I execute the queries. I.e.: if I execute query1 before and query2 after, it crashes. If I execute the same exact query2 before and query1 after, it works. The crash occours only if, after db.Query(...), I read the data using rows.Next(). If I just execute the db.Query(...), it does not crash. Can you help me diagnose the issue? Stack is:
|
My problem is indeed a duplicate #18, which is in turn a duplicate of golang/go#9457: adding |
At the first, I don't still reproduce this. So I can't put the workaround in README.md . |
Anyway, it seems a way to save my (or your's) time. updated README. Thanks. |
I have divided the mdb.go example in two apps:
When I test the mdb2 app manually with the browser, curl, etc it works fine, but when y put them on load, for example:
ab -n 100 -c 10 -k http://app.url/
ab exits with "...apr_socket_recv: Connection reset by peer (54)", and the app crashes with:
I have test it wit wrk with similar errors:
wrk -t2 -c10 -d5s http://app.url
Thanks.
The text was updated successfully, but these errors were encountered: