Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry logic can cause queries to be silently performed twice #939

Open
jackc opened this issue Feb 1, 2020 · 1 comment
Open

Retry logic can cause queries to be silently performed twice #939

jackc opened this issue Feb 1, 2020 · 1 comment

Comments

@jackc
Copy link

jackc commented Feb 1, 2020

driver.ErrBadConn should only be returned when it is safe to retry the query. lib/pq can return this error when the query has already been sent.

I ran across this because someone was requesting graceful handling of database restarts in jackc/pgx#672. To me that is a very tricky problem to handle without sacrificing either safety or performance. But they said that lib/pq does what they want. So I took a quick look and it appears that safety is being sacrificed.

Below is an example. It runs a query that only inserts one record, but there are 2 records inserted because internally it is retried.

package main

import (
	"database/sql"
	"log"
	"os"

	_ "github.com/lib/pq"
)

func main() {
	db, err := sql.Open("postgres", os.Getenv("DATABASE_URL"))
	if err != nil {
		log.Fatal(err)
	}

	_, err = db.Exec("drop table if exists t")
	if err != nil {
		log.Fatal(err)
	}

	_, err = db.Exec("create table t (id int primary key generated by default as identity)")
	if err != nil {
		log.Fatal(err)
	}

	_, err = db.Exec(`
begin;
insert into t(id) values (default);
commit;
select pg_terminate_backend(pg_backend_pid()) where 1 = (select count(*) from t);
`)
	if err != nil {
		log.Fatal(err)
	}

	var numInserts int
	err = db.QueryRow("select count(*) from t").Scan(&numInserts)
	if err != nil {
		log.Fatal(err)
	}

	log.Println("expected 1 row inserted, got", numInserts)
}
jack@glados ~/dev/pqretrybug » PGHOST=/private/tmp go run main.go
2020/02/01 15:56:40 expected 1 row inserted, got 2
@andreimatei
Copy link

I think this issues was supposed to have been fixed in #730 - but apparently it hasn't?
CC @mjibson

storjBuildBot pushed a commit to storj/storj that referenced this issue Jul 13, 2020
What:

Use the github.com/jackc/pgx postgresql driver in place of
github.com/lib/pq.

Why:

github.com/lib/pq has some problems with error handling and context
cancellations (i.e. it might even issue queries or DML statements more
than once! see lib/pq#939). The
github.com/jackx/pgx library appears not to have these problems, and
also appears to be better engineered and implemented (in particular, it
doesn't use "exceptions by panic"). It should also give us some
performance improvements in some cases, and even more so if we can use
it directly instead of going through the database/sql layer.

Change-Id: Ia696d220f340a097dee9550a312d37de14ed2044
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants