Switch branches/tags
Find file
Fetching contributors…
Cannot retrieve contributors at this time
771 lines (648 sloc) 28.5 KB
<html xmlns="">
<title>CL-postgres reference manual</title>
<link rel="stylesheet" type="text/css" href="style.css"/>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
<h1>CL-postgres reference manual</h1>
<p>The CL-postgres module implements a rather low-level interface
for communicating with a <a
href="">PostgreSQL</a> database server. It is
part of the <a href="index.html">Postmodern</a> library, but can
be used separately.</p>
<li><a href="#connecting">Connecting</a></li>
<li><a href="#querying">Querying</a></li>
<li><a href="#reading">Reading values</a></li>
<li><a href="#row-readers">Row readers</a></li>
<li><a href="#bulk-copying">Bulk Copying</a></li>
<li><a href="#conditions">Conditions</a></li>
<li><a href="#index">Symbol-index</a></li>
<h2><a name="connecting">Connecting</a></h2>
<p class="def">
<a name="database-connection"></a>
<p class="desc">Objects of this type represent database
<p class="def">
<a name="open-database"></a>
open-database (database user password host &amp;optional (port 5432) (use-ssl :no))
<br/>&#8594; database-connection
<p class="desc">Create and open a connection for the specified
server, database, and user. <code>use-ssl</code> may be
<code>:no</code>, <code>:yes</code>, or <code>:try</code>, where
<code>:try</code> means 'if the server supports it'. When it is
anything but <code>:no</code>, you must have the <a
package loaded to initiate the connection.</p>
<p class="desc">On SBCL and Clozure CL, the value
<code>:unix</code> may be passed for <code>host</code>, in order
to connect using a Unix domain socket instead of a TCP socket.</p>
<p class="def">
<a name="close-database"></a>
close-database (database-connection)
<p class="desc">Close a database connection. It is advisable to
call this on connections when you are done with them. Otherwise
the open socket will stick around until it is garbage collected,
and no one will tell the database server that we are done
with it.</p>
<p class="def">
<a name="reopen-database"></a>
reopen-database (database-connection)
<p class="desc">Re-establish a database connection for a
previously closed connection object. (Calling this on a connection
that is still open is harmless.)</p>
<p class="def">
<a name="database-open-p"></a>
database-open-p (database-connection)
<br/>&#8594; boolean
<p class="desc">Test whether a database connection is still
<p class="def">
<a name="connection-meta"></a>
connection-meta (database-connection)
<br/>&#8594; hash-table
<p class="desc">This method provides access to a hash table that
is associated with the current database connection. When the
connection is closed and re-opened this hash table is reset. The
most obvious use for this is for storing information about the
prepared statements that have been parsed for this connection.</p>
<p class="def">
<a name="connection-parameters"></a>
connection-parameters (database-connection)
<br/>&#8594; hash-table
<p class="desc">This method returns a mapping (string to string)
containing all the <a
parameters</a> for the connection.</p>
<p class="def">
<a name="*unix-socket-dir*"></a>
<p class="desc">On SBCL, when using the <code>:unix</code> keyword
as host argument when creating a connection, this variable
determines the directory in which CL-Postgres will look for the
socket file.</p>
<p class="def">
<a name="*ssl-certificate-file*"></a>
<a name="*ssl-key-file*"></a>
<p class="desc">When using SSL (see <a
href="#open-database"><code>open-database</code></a>), these can
be used to provide client key and certificate files. They can be
either <code>NIL</code>, for no file, or a pathname.</p>
<p class="def">
<a name="wait-for-notification"></a>
wait-for-notification (database-connection)
<p class="desc">This function blocks until a notification is
received on the connection. The PostgreSQL <code>LISTEN</code>
command must be used to enable listening for notifications.</p>
<h2><a name="querying">Querying</a></h2>
<p class="def">
<a name="exec-query"></a>
exec-query (database-connection query &amp;optional (row-reader '<a href="#ignore-row-reader"><code>ignore-row-reader</code></a>))
<br/>&#8594; result
<p class="desc">Sends the given query to the given connection, and
interprets the results (if there are any) with the given <a
href="#reading">row-reader</a>. If the database returns
information about the amount of rows affected, this is returned as
a second value.</p>
<p class="def">
<a name="prepare-query"></a>
prepare-query (database-connection name query)
<p class="desc">Parse and plan the given query, and store it under
the given name. Note that prepared statements are per-connection,
so they can only be executed through the same connection that
prepared them.</p>
<p class="def">
<a name="exec-prepared"></a>
exec-prepared (database-connection name parameters &amp;optional (row-reader '<a href="#ignore-row-reader"><code>ignore-row-reader</code></a>))
<br/>&#8594; result
<p class="desc">Execute the prepared statement by the given name.
Parameters should be given as a list. Each value in this list
should be of a type that <a
href="#to-sql-string"><code>to-sql-string</code></a> has been
specialised on. (Byte arrays will be passed in their binary form,
without being put through <a
href="#to-sql-string"><code>to-sql-string</code></a>.) The result
of the executing the statement, if any, is interpreted by the
given <a href="#row-readers">row reader</a>, and returned. Again,
the number or affected rows is optionally returned as a second
<p class="def">
<a name="unprepare-query"></a>
unprepare-query (database-connection name)
<p class="desc">Close the prepared statement by the given name.
This will free resources and allow the name to be associated with
a new prepared query.</p>
<p class="def">
<a name="to-sql-string"></a>
to-sql-string (value)
<br/>&#8594; (values string needs-escaping)
<p class="desc">Convert a Lisp value to its textual
<em>unescaped</em> SQL representation. Returns a second value
indicating whether this value should be escaped if it is to be put
directly into a query.</p>
<p class="desc">You can define <code>to-sql-string</code> methods
for your own datatypes if you want to be able to pass them to <a
href="#exec-prepared"><code>exec-prepared</code></a>. When a
non-<code>NIL</code> second value is returned, this may be
<code>T</code> to indicate that the first value should simply be
escaped as a string, or a second string providing a type prefix
for the value. (This is used by <a
<p class="def">
<a name="*silently-truncate-rationals*"></a>
<p class="desc">When a rational number is passed into a query (as
per <a href="#to-sql-string"><code>to-sql-string</code></a>), but
it can not be expressed within 38 decimal digits (for example
<code>1/3</code>), it will be truncated, and lose some precision.
Set this variable to <code>nil</code> to suppress that behaviour
and raise an error instead.</p>
<p class="def">
<a name="*query-log*"></a>
<p class="desc">When debugging, it can be helpful to inspect the
queries that are being sent to the database. Set this variable to
an output stream value (<code>*standard-output*</code>, for
example) to have CL-postgres log every query it makes.</p>
<p class="def">
<a name="*query-callback*"></a>
<p class="desc">
When profiling or debugging,
the <a href="#*query-log*"><code>*query-log*</code></a> may not
give enough information, or reparsing its output may not be
feasible. This variable may be set to a designator of function
taking two arguments. This function will be then called after
every query, and receive query string and internal time units
(as in <code>(CL:GET-INTERNAL-REAL-TIME)</code>) spent in query
as its arguments.
<p class="desc">
Default value of this variable
is <a href="#log-query"><code>'LOG-QUERY</code></a>, which takes
care of <a href="#*query-log*"><code>*QUERY-LOG*</code></a>
processing. If you provide custom query callback and wish to
keep <a href="#*query-log*"><code>*QUERY-LOG*</code></a>
functionality, you will have to
call <a href="#log-query"><code>LOG-QUERY</code></a> from your
callback function
<p class="def">
<a name="log-query"></a>
log-query (query internal-time)
<p class="desc">
This function is default value
of <a href="#*query-callback*"><code>*QUERY-CALLBACK*</code></a>
and logs queries
to <a href="#*query-log*"><code>*QUERY-LOG*</code></a> if it is
not <code>NIL</code>.
<h2><a name="reading">Reading values</a></h2>
<p>CL-postgres knows how to convert commonly used PostgreSQL data
types to Lisp values. This table shows the mapping:</p>
<tr><td>double precision</td><td>double-float</td></tr>
<tr><td>bytea</td><td>(vector (unsigned-byte 8))</td></tr>
<p>The mapping from PostgreSQL types (identified by OID numbers)
to the functions that interpret them is kept in so-called SQL
readtables. All types for which no reader is defined will be
returned as string values containing their PostgreSQL
<p class="def">
<a name="*sql-readtable*"></a>
<p class="desc">This variable is used to choose the current
readtable. For simple use, you will not have to touch this, but it
is possible that code within a Lisp image requires different
readers in different situations, in which case you can create
separate read tables.</p>
<p class="def">
<a name="copy-sql-readtable"></a>
copy-sql-readtable (table)
<br/>&#8594; readtable
<p class="desc">Copies a given readtable.</p>
<p class="def">
<a name="default-sql-readtable"></a>
default-sql-readtable ()
<br/>&#8594; readtable
<p class="desc">Returns the default readtable, containing only the
readers defined by CL-postgres itself.</p>
<p class="def">
<a name="set-sql-reader"></a>
set-sql-reader (oid function &amp;key table binary-p)
<p class="desc">Define a new reader for a given type.
<code>table</code> defaults to <a
href="#*sql-readtable"><code>*sql-readtable*</code></a>. The
reader function should take a single argument, a string, and
transform that into some kind of equivalent Lisp value. When
<code>binary-p</code> is true, the reader function is supposed to
directly read the binary representation of the value. In most
cases this is not recommended, but if you want to use it: provide
a function that takes a binary input stream and an integer (the
size of the value, in bytes), and reads the value from that
stream. Note that reading less or more bytes than the given size
will horribly break your connection.</p>
<p class="def">
<a name="set-sql-datetime-readers"></a>
set-sql-datetime-readers (&amp;key date timestamp timestamp-with-timezone time interval table)
<p class="desc">Since there is no widely recognised standard way
of representing dates and times in Common Lisp, and reading these
from string representation is clunky and slow, this function
provides a way to easily plug in binary readers for the
<code>date</code>, <code>time</code>, <code>timestamp</code>, and
<code>interval</code> types. It should be given functions with the
following signatures:</p>
<dl class="desc">
<dt><code>:date (days)</code></dt>
<dd>Where <code>days</code> is the amount of days since January
1st, 2000.</dd>
<dt><code>:timestamp (useconds)</code></dt>
<dd>Timestamps have a microsecond resolution. Again, the zero
point is the start of the year 2000, UTC.</dd>
<dd>Like <code>:timestamp</code>, but for values of the
'timestamp with time zone' type (which PostgreSQL internally
stores exactly the same as regular timestamps).</dd>
<dt><code>:time (useconds)</code></dt>
<dd>Refers to a time of day, counting from midnight.</dd>
<dt><code>:interval (months days useconds)</code></dt>
<dd>An interval is represented as several separate components.
The reason that days and microseconds are separated is that you
might want to take leap seconds into account.</dd>
<h2><a name="row-readers"></a>Row readers</h2>
<p>Row readers are a way to read and group the results of queries.
Roughly, they are functions that perform the iteration over the
rows and cells in the result, and do <em>something</em> with the
returned values.</p>
<p class="def">
<a name="row-reader"></a>
row-reader ((fields) &amp;body body)
<br/>&#8594; function
<p class="desc">Creates a row-reader, using the given name for the
variable. Inside the body this variable refers to a vector of
field descriptions. On top of that, two local functions are bound,
<code>next-row</code> and <code>next-field</code>. The first will
start reading the next row in the result, and returns a boolean
indicating whether there is another row. The second will read and
return one field, and should be passed the corresponding field
description from the fields argument as a parameter.</p>
<p class="desc">A row reader should take care to iterate over all
the rows in a result, and within each row iterate over all the
fields. This means it should contain an outer loop that calls
<code>next-row</code>, and every time <code>next-row</code>
returns <code>T</code> it should iterate over the fields vector
and call <code>next-field</code> for every field.</p>
<p class="desc">The definition of <a
href="#list-row-reader"><code>list-row-reader</code></a> should
give you an idea what a row reader looks like:</p>
<pre class="desc code">
(row-reader (fields)
(loop :while (next-row)
:collect (loop :for field :across fields
:collect (next-field field))))</pre>
<p class="desc">Obviously, row readers should <em>not</em> do
things with the database connection like, say, close it or start a
new query, since it still reading out the results from the current
<p class="def">
<a name="def-row-reader"></a>
def-row-reader (name (fields) &amp;body body)
<p class="desc">The <code>defun</code>-like variant of <a
href="#row-reader"><code>row-reader</code></a>: creates a row
reader and gives it a top-level function name.</p>
<p class="def">
<a name="field-name"></a>
field-name (field)
<br/>&#8594; string
<p class="desc">This can be used to get information about the
fields read by a row reader. Given a field description, it returns
the name the database associated with this column.</p>
<p class="def">
<a name="field-type"></a>
field-type (field)
<br/>&#8594; oid
<p class="desc">This extracts the PostgreSQL <a
associated with this column. You can, if you really want to, query
the pg_types table to find out more about the types denoted by
<p class="def">
<a name="list-row-reader"></a>
list-row-reader (socket fields)
<br/>&#8594; list
<p class="desc">A row reader that builds a list of lists from the
query results.</p>
<p class="def">
<a name="alist-row-reader"></a>
alist-row-reader (socket fields)
<br/>&#8594; alist
<p class="desc">A row reader that returns a list of alists, which
associate column names with values.</p>
<p class="def">
<a name="ignore-row-reader"></a>
ignore-row-reader (socket fields)
<p class="desc">A row reader that completely ignores the result of
a query.</p>
<h2><a name="bulk-copying">Bulk Copying</a></h2>
<p>When loading large amounts of data into PostgreSQL, it can be done
significantly faster using the bulk copying feature. The drawback to
this approach is that you don't find out about data integrity errors
until the entire batch is completed but sometimes the speed is worth it
<p class="def">
<a name="open-db-writer"></a>
open-db-writer (db table &optional columns)
<p class="desc">Opens a table stream into which rows can be
written one at a time
using <code>db-write-row</code>. <code>db</code> is either
a <a href="#database-connection">connection</a> object or a list
of arguments that could be passed to <code>open-database</code>.
<code>table</code> is the name of an existing table into which this writer
will write rows. If you don't have data for all columns, use
<code>columns</code> to indicate those that you do.
<p class="def">
<a name="close-db-writer"></a>
close-db-writer (writer &key abort)
<p class="desc">Closes a bulk writer opened
by <code>open-db-writer</code>. Will close the associated database
connection when it was created for this copier,
or <code>abort</code> is true.</p>
<p class="def">
<a name="db-write-row"></a>
db-write-row (writer row-data)
<p class="desc">Writes <code>row-data</code> into the table and columns
referenced by the writer. <code>row-data</code> is a list of Lisp objects,
one for each column included when opening the writer. Arrays (the elements
of which must all be the same type) will be serialized into their PostgreSQL
representation before being written into the DB.
<h2><a name="conditions">Conditions</a></h2>
<p>Opening or querying a database may raise errors. CL-postgres
will wrap the errors that the server returns in a lisp condition,
and raise conditions of the same type when it detects some problem
itself. Socket errors are let through as they are.</p>
<p class="def">
<a name="database-error"></a>
<p class="desc">The type of database-related conditions. For
errors that you may want to catch by type, the
<code>cl-postgres-error</code> package defines a bucket of
subtypes used for specific errors. See the
<code>cl-postgres/package.lisp</code> file for a list.</p>
<p class="def">
<a name="database-error-message"></a>
database-error-message (database-error)
<br/>&#8594; string
<p class="desc">A short message associated with this error.</p>
<p class="def">
<a name="database-error-detail"></a>
database-error-detail (database-error)
<br/>&#8594; string
<p class="desc">A longer description of the problem, or
<code>NIL</code> if none is available.</p>
<p class="def">
<a name="database-error-code"></a>
database-error-code (database-error)
<br/>&#8594; string
<p class="desc">The error code PostgreSQL associated with this
error, if any. See the <a
manual</a> for their meaning.</p>
<p class="def">
<a name="database-error-query"></a>
database-error-query (database-error)
<br/>&#8594; string
<p class="desc">The query that led to this error, or
<code>NIL</code> if no query was involved.</p>
<p class="def">
<a name="database-error-cause"></a>
database-error-cause (database-error)
<br/>&#8594; condition
<p class="desc">The condition that caused this error, or
<code>NIL</code> when it was not caused by another condition.</p>
<p class="def">
<a name="database-error-constraint-name"></a>
database-error-constraint-name (database-error)
<br/>&#8594; string
<p class="desc">For integrity-violation errors, returns the name
of the constraint that was violated (or <code>nil</code> if no
constraint was found.)</p>
<p class="def">
<a name="database-connection-error"></a>
<p class="desc">Subtype of <a
href="#database-error"><code>database-error</code></a>. An error
of this type (or one of its subclasses) is signaled when a query
is attempted with a connection object that is no longer connected,
or a database connection becomes invalid during a query. Always
provides a <code>:reconnect</code> restart, which will cause the
library to make an attempt to restore the connection and re-try
the query.</p>
<p class="desc">The following shows an example use of this
feature, a way to ensure that the first connection error causes a
reconnect attempt, while others pass through as normal. A
variation on this theme could continue trying to reconnect, with
successively longer pauses.</p>
<pre class="desc code">
(defun call-with-single-reconnect (fun)
(let ((reconnected nil))
(lambda (err)
(when (not reconnected)
(setf reconnected t)
(invoke-restart :reconnect)))))
(funcall fun))))</pre>
<p class="def">
<a name="postgresql-notification"></a>
<p class="desc">The condition that is signalled when a
notification message is received from the PostgreSQL server.
This is a <code>WARNING</code> condition which is caught by
the <code>WAIT-FOR-NOTIFICATION</code> function that implements
synchronous waiting for notifications.</p>
<p class="def">
<a name="postgresql-notification-channel"></a>
postgresql-notification-channel (postgresql-notification)
<br/>&#8594; string
<p class="desc">The channel string of this notification.</p>
<p class="def">
<a name="postgresql-notification-payload"></a>
postgresql-notification-payload (postgresql-notification)
<br/>&#8594; string
<p class="desc">The payload of this notification.</p>
<p class="def">
<a name="postgresql-notification-pid"></a>
postgresql-notification-pid (postgresql-notification)
<br/>&#8594; integer
<p class="desc">The process ID of the process that sent the
<h2><a name="index">Symbol-index</a></h2>
<ul class="symbol-index">
<li><a href="#alist-row-reader">alist-row-reader</a></li>
<li><a href="#close-database">close-database</a></li>
<li><a href="#close-db-writer">close-db-writer</a></li>
<li><a href="#connection-meta">connection-meta</a></li>
<li><a href="#connection-parameters">connection-parameters</a></li>
<li><a href="#copy-sql-readtable">copy-sql-readtable</a></li>
<li><a href="#database-connection">database-connection</a></li>
<li><a href="#database-connection-error">database-connection-error</a></li>
<li><a href="#database-error">database-error</a></li>
<li><a href="#database-error-cause">database-error-cause</a></li>
<li><a href="#database-error-code">database-error-code</a></li>
<li><a href="#database-error-constraint-name">database-error-constraint-name</a></li>
<li><a href="#database-error-detail">database-error-detail</a></li>
<li><a href="#database-error-message">database-error-message</a></li>
<li><a href="#database-error-query">database-error-query</a></li>
<li><a href="#database-open-p">database-open-p</a></li>
<li><a href="#def-row-reader">def-row-reader</a></li>
<li><a href="#default-sql-readtable">default-sql-readtable</a></li>
<li><a href="#exec-prepared">exec-prepared</a></li>
<li><a href="#exec-query">exec-query</a></li>
<li><a href="#field-name">field-name</a></li>
<li><a href="#field-type">field-type</a></li>
<li><a href="#ignore-row-reader">ignore-row-reader</a></li>
<li><a href="#list-row-reader">list-row-reader</a></li>
<li><a href="#log-query">log-query</a></li>
<li><a href="#row-reader">next-field</a></li>
<li><a href="#row-reader">next-row</a></li>
<li><a href="#open-database">open-database</a></li>
<li><a href="#open-db-writer">open-db-writer</a></li>
<li><a href="#postgresql-notification">postgresql-notification</a></li>
<li><a href="#postgresql-notification-channel">postgresql-notification-channel</a></li>
<li><a href="#postgresql-notification-payload">postgresql-notification-payload</a></li>
<li><a href="#postgresql-notification-pid">postgresql-notification-pid</a></li>
<li><a href="#prepare-query">prepare-query</a></li>
<li><a href="#*query-callback*">*query-callback*</a></li>
<li><a href="#*query-log*">*query-log*</a></li>
<li><a href="#reopen-database">reopen-database</a></li>
<li><a href="#row-reader">row-reader</a></li>
<li><a href="#set-sql-datetime-readers">set-sql-datetime-readers</a></li>
<li><a href="#set-sql-reader">set-sql-reader</a></li>
<li><a href="#write-db-row">write-db-row</a></li>
<li><a href="#*silently-truncate-rationals*">*silently-truncate-rationals*</a></li>
<li><a href="#*sql-readtable*">*sql-readtable*</a></li>
<li><a href="#*ssl-certificate-file*">*ssl-certificate-file*</a></li>
<li><a href="#*ssl-key-file*">*ssl-key-file*</a></li>
<li><a href="#to-sql-string">to-sql-string</a></li>
<li><a href="#*unix-socket-dir*">*unix-socket-dir*</a></li>
<li><a href="#wait-for-notification">wait-for-notification</a></li>