PostgreSQL driver and toolkit for Go
jackc Merge pull request #480 from gcurtis/race
stdlib: fix race with Driver.configs in Open
Latest commit bd37aaa Nov 10, 2018
Permalink
Failed to load latest commit information.
chunkreader Reduce allocations and copies in pgproto3 Apr 29, 2017
examples Fix go vet issues Jan 13, 2018
internal/sanitize Use github.com/pkg/errors Jun 5, 2017
log Small typo Mar 21, 2018
pgio Remove read functions from pgio and update docs May 6, 2017
pgmock Type modifier should be int32 not uint32 Sep 22, 2018
pgproto3 Type modifier should be int32 not uint32 Sep 22, 2018
pgtype Merge pull request #454 from regeda/macaddr-array Sep 2, 2018
stdlib stdlib: fix race with Driver.configs in Open Nov 10, 2018
travis Add domain support Apr 7, 2018
.gitignore Introduce pgproto3 package Apr 29, 2017
.travis.yml Postgres 9.2 is not supported since September 2017 Dec 17, 2017
CHANGELOG.md Update changelog Aug 7, 2018
LICENSE Add license Jul 31, 2013
README.md Add domain support Apr 7, 2018
batch.go Add docs regarding Batch.Send deadlock potential Jan 15, 2018
batch_test.go Fix go vet issues Jan 13, 2018
bench-tmp_test.go Initial proof-of-concept for pgtype Mar 3, 2017
bench_test.go Finish core batch operations Jun 3, 2017
conn.go Merge pull request #477 from fastest963/cancel Nov 9, 2018
conn_config_test.go.example Add a clientcert example to simplify future TLS testing. Feb 2, 2018
conn_config_test.go.travis travis: add connection test coverage for cratedb Nov 12, 2017
conn_pool.go Added CheckedOutConnections for ConnPoolStat Oct 23, 2018
conn_pool_private_test.go Make ConnPool.Acquire() non blocking, feedback Jul 6, 2016
conn_pool_test.go ConnPool begin should not retry if ctx is done Nov 21, 2017
conn_test.go Add host query parameter to support unix sockets Nov 3, 2018
copy_from.go Fix: Handle (n > 0 and err == io.EOF) in CopyFromReader as per io.Rea… Oct 8, 2018
copy_from_test.go Fix: Handle (n > 0 and err == io.EOF) in CopyFromReader as per io.Rea… Oct 8, 2018
copy_to.go addressing the comments Aug 7, 2018
copy_to_test.go addressing the comments Aug 7, 2018
doc.go tls: stop sending ssl_renegotiation_limit in startup message Oct 30, 2018
example_custom_type_test.go Use github.com/pkg/errors Jun 5, 2017
example_json_test.go Remove skip test for missing json type Mar 3, 2017
fastpath.go Fix fastpath / largeobjects query counting Mar 31, 2018
go_stdlib.go Support nil pointers to value implementing driver.Valuer Oct 27, 2017
helper_test.go Happy-path batch query mode Jun 2, 2017
large_objects.go Use Go casing convention for OID Jun 3, 2017
large_objects_test.go Fix fastpath / largeobjects query counting Mar 31, 2018
logger.go Use github.com/pkg/errors Jun 5, 2017
messages.go add float types to database/sql ColumnType ScanType() Sep 22, 2018
pgpass.go Support pgpass Mar 17, 2017
pgpass_test.go Run gofmt with simplify flag Dec 21, 2017
private_test.go Remove Conn.TxStatus Jan 7, 2017
query.go Fix incorrect reuse of Value in Rows.Values() Feb 3, 2018
query_test.go Fix incorrect reuse of Value in Rows.Values() Feb 3, 2018
replication.go Replication: add NOEXPORT_SNAPSHOT, add GetConnInfo() func Oct 23, 2018
replication_test.go Reverted breaking API change to CreateReplicationSlot. Instead, the n… Sep 21, 2017
sql.go Cleanups Aug 30, 2016
sql_test.go Cleanups Aug 30, 2016
stress_test.go Call context.WithTimeout cancel function Sep 15, 2018
tx.go Call context.WithTimeout cancel function Sep 15, 2018
tx_test.go Call context.WithTimeout cancel function Sep 15, 2018
values.go Convert driver.Valuer's earlier in bind path Sep 2, 2018
values_test.go Removes duplicate if/else Sep 25, 2017

README.md

Build Status

pgx - PostgreSQL Driver and Toolkit

pgx is a pure Go driver and toolkit for PostgreSQL. pgx is different from other drivers such as pq because, while it can operate as a database/sql compatible driver, pgx is also usable directly. It offers a native interface similar to database/sql that offers better performance and more features.

var name string
var weight int64
err := conn.QueryRow("select name, weight from widgets where id=$1", 42).Scan(&name, &weight)
if err != nil {
    return err
}

Features

pgx supports many additional features beyond what is available through database/sql.

  • Support for approximately 60 different PostgreSQL types
  • Batch queries
  • Single-round trip query mode
  • Full TLS connection control
  • Binary format support for custom types (can be much faster)
  • Copy protocol support for faster bulk data loads
  • Extendable logging support including built-in support for log15 and logrus
  • Connection pool with after connect hook to do arbitrary connection setup
  • Listen / notify
  • PostgreSQL array to Go slice mapping for integers, floats, and strings
  • Hstore support
  • JSON and JSONB support
  • Maps inet and cidr PostgreSQL types to net.IPNet and net.IP
  • Large object support
  • NULL mapping to Null* struct or pointer to pointer.
  • Supports database/sql.Scanner and database/sql/driver.Valuer interfaces for custom types
  • Logical replication connections, including receiving WAL and sending standby status updates
  • Notice response handling (this is different than listen / notify)

Performance

pgx performs roughly equivalent to go-pg and is almost always faster than pq. When parsing large result sets the percentage difference can be significant (16483 queries/sec for pgx vs. 10106 queries/sec for pq -- 63% faster).

In many use cases a significant cause of latency is network round trips between the application and the server. pgx supports query batching to bundle multiple queries into a single round trip. Even in the case of a connection with the lowest possible latency, a local Unix domain socket, batching as few as three queries together can yield an improvement of 57%. With a typical network connection the results can be even more substantial.

See this gist for the underlying benchmark results or checkout go_db_bench to run tests for yourself.

In addition to the native driver, pgx also includes a number of packages that provide additional functionality.

github.com/jackc/pgx/stdlib

database/sql compatibility layer for pgx. pgx can be used as a normal database/sql driver, but at any time the native interface may be acquired for more performance or PostgreSQL specific functionality.

github.com/jackc/pgx/pgtype

Approximately 60 PostgreSQL types are supported including uuid, hstore, json, bytea, numeric, interval, inet, and arrays. These types support database/sql interfaces and are usable even outside of pgx. They are fully tested in pgx and pq. They also support a higher performance interface when used with the pgx driver.

github.com/jackc/pgx/pgproto3

pgproto3 provides standalone encoding and decoding of the PostgreSQL v3 wire protocol. This is useful for implementing very low level PostgreSQL tooling.

github.com/jackc/pgx/pgmock

pgmock offers the ability to create a server that mocks the PostgreSQL wire protocol. This is used internally to test pgx by purposely inducing unusual errors. pgproto3 and pgmock together provide most of the foundational tooling required to implement a PostgreSQL proxy or MitM (such as for a custom connection pooler).

Documentation

pgx includes extensive documentation in the godoc format. It is viewable online at godoc.org.

Testing

pgx supports multiple connection and authentication types. Setting up a test environment that can test all of them can be cumbersome. In particular, Windows cannot test Unix domain socket connections. Because of this pgx will skip tests for connection types that are not configured.

Normal Test Environment

To setup the normal test environment, first install these dependencies:

go get github.com/cockroachdb/apd
go get github.com/hashicorp/go-version
go get github.com/jackc/fake
go get github.com/lib/pq
go get github.com/pkg/errors
go get github.com/satori/go.uuid
go get github.com/shopspring/decimal
go get github.com/sirupsen/logrus
go get go.uber.org/zap
go get gopkg.in/inconshreveable/log15.v2

Then run the following SQL:

create user pgx_md5 password 'secret';
create user " tricky, ' } "" \ test user " password 'secret';
create database pgx_test;
create user pgx_replication with replication password 'secret';

Connect to database pgx_test and run:

create extension hstore;
create domain uint64 as numeric(20,0);

Next open conn_config_test.go.example and make a copy without the .example. If your PostgreSQL server is accepting connections on 127.0.0.1, then you are done.

Connection and Authentication Test Environment

Complete the normal test environment setup and also do the following.

Run the following SQL:

create user pgx_none;
create user pgx_pw password 'secret';

Add the following to your pg_hba.conf:

If you are developing on Unix with domain socket connections:

local  pgx_test  pgx_none  trust
local  pgx_test  pgx_pw    password
local  pgx_test  pgx_md5   md5

If you are developing on Windows with TCP connections:

host  pgx_test  pgx_none  127.0.0.1/32 trust
host  pgx_test  pgx_pw    127.0.0.1/32 password
host  pgx_test  pgx_md5   127.0.0.1/32 md5

Replication Test Environment

Add a replication user:

create user pgx_replication with replication password 'secret';

Add a replication line to your pg_hba.conf:

host replication pgx_replication 127.0.0.1/32 md5

Change the following settings in your postgresql.conf:

wal_level=logical
max_wal_senders=5
max_replication_slots=5

Set replicationConnConfig appropriately in conn_config_test.go.

Version Policy

pgx follows semantic versioning for the documented public API on stable releases. Branch v3 is the latest stable release. master can contain new features or behavior that will change or be removed before being merged to the stable v3 branch (in practice, this occurs very rarely). v2 is the previous stable release.