sqlite3
Gerhard Häring <gh@ghaering.de>
Source code: Lib/sqlite3/
import sqlite3 src = sqlite3.connect(":memory:", isolation_level=None) dst = sqlite3.connect("tutorial.db", isolation_level=None) src.backup(dst) del src, dst
SQLite is a C library that provides a lightweight disk-based database that doesn't require a separate server process and allows accessing the database using a nonstandard variant of the SQL query language. Some applications can use SQLite for internal data storage. It's also possible to prototype an application using SQLite and then port the code to a larger database such as PostgreSQL or Oracle.
The !sqlite3
module was written by Gerhard Häring. It provides an SQL interface compliant with the DB-API 2.0 specification described by 249
, and requires SQLite 3.7.15 or newer.
This document includes four main sections:
sqlite3-tutorial
teaches how to use the!sqlite3
module.sqlite3-reference
describes the classes and functions this module defines.sqlite3-howtos
details how to handle specific tasks.sqlite3-explanation
provides in-depth background on transaction control.
- https://www.sqlite.org
The SQLite web page; the documentation describes the syntax and the available data types for the supported SQL dialect.
- https://www.w3schools.com/sql/
Tutorial, reference and examples for learning SQL syntax.
249
- Database API Specification 2.0PEP written by Marc-André Lemburg.
In this tutorial, you will create a database of Monty Python movies using basic !sqlite3
functionality. It assumes a fundamental understanding of database concepts, including cursors and transactions.
First, we need to create a new database and open a database connection to allow !sqlite3
to work with it. Call sqlite3.connect
to to create a connection to the database tutorial.db
in the current working directory, implicitly creating it if it does not exist:
import sqlite3 con = sqlite3.connect("tutorial.db")
The returned Connection
object con
represents the connection to the on-disk database.
In order to execute SQL statements and fetch results from SQL queries, we will need to use a database cursor. Call con.cursor() <Connection.cursor>
to create the Cursor
:
cur = con.cursor()
Now that we've got a database connection and a cursor, we can create a database table movie
with columns for title, release year, and review score. For simplicity, we can just use column names in the table declaration --thanks to the flexible typing feature of SQLite, specifying the data types is optional. Execute the CREATE TABLE
statement by calling cur.execute(...) <Cursor.execute>
:
cur.execute("CREATE TABLE movie(title, year, score)")
We can verify that the new table has been created by querying the sqlite_master
table built-in to SQLite, which should now contain an entry for the movie
table definition (see The Schema Table for details). Execute that query by calling cur.execute(...) <Cursor.execute>
, assign the result to res
, and call res.fetchone() <Cursor.fetchone>
to fetch the resulting row:
>>> res = cur.execute("SELECT name FROM sqlite_master") >>> res.fetchone() ('movie',)
We can see that the table has been created, as the query returns a tuple
containing the table's name. If we query sqlite_master
for a non-existent table spam
, !res.fetchone()
will return None
:
>>> res = cur.execute("SELECT name FROM sqlite_master WHERE name='spam'") >>> res.fetchone() is None True
Now, add two rows of data supplied as SQL literals by executing an INSERT
statement, once again by calling cur.execute(...) <Cursor.execute>
:
- cur.execute("""
- INSERT INTO movie VALUES
('Monty Python and the Holy Grail', 1975, 8.2), ('And Now for Something Completely Different', 1971, 7.5)
""")
The INSERT
statement implicitly opens a transaction, which needs to be committed before changes are saved in the database (see sqlite3-controlling-transactions
for details). Call con.commit() <Connection.commit>
on the connection object to commit the transaction:
con.commit()
We can verify that the data was inserted correctly by executing a SELECT
query. Use the now-familiar cur.execute(...) <Cursor.execute>
to assign the result to res
, and call res.fetchall() <Cursor.fetchall>
to return all resulting rows:
>>> res = cur.execute("SELECT score FROM movie") >>> res.fetchall() [(8.2,), (7.5,)]
The result is a list
of two !tuple
s, one per row, each containing that row's score
value.
Now, insert three more rows by calling cur.executemany(...) <Cursor.executemany>
:
- data = [
("Monty Python Live at the Hollywood Bowl", 1982, 7.9), ("Monty Python's The Meaning of Life", 1983, 7.5), ("Monty Python's Life of Brian", 1979, 8.0),
] cur.executemany("INSERT INTO movie VALUES(?, ?, ?)", data) con.commit() # Remember to commit the transaction after executing INSERT.
Notice that ?
placeholders are used to bind data
to the query. Always use placeholders instead of string formatting <tut-formatting>
to bind Python values to SQL statements, to avoid SQL injection attacks (see sqlite3-placeholders
for more details).
We can verify that the new rows were inserted by executing a SELECT
query, this time iterating over the results of the query:
>>> for row in cur.execute("SELECT year, title FROM movie ORDER BY year"): ... print(row) (1971, 'And Now for Something Completely Different') (1975, 'Monty Python and the Holy Grail') (1979, "Monty Python's Life of Brian") (1982, 'Monty Python Live at the Hollywood Bowl') (1983, "Monty Python's The Meaning of Life")
Each row is a two-item tuple
of (year, title)
, matching the columns selected in the query.
Finally, verify that the database has been written to disk by calling con.close() <Connection.close>
to close the existing connection, opening a new one, creating a new cursor, then querying the database:
>>> con.close() >>> new_con = sqlite3.connect("tutorial.db") >>> new_cur = new_con.cursor() >>> res = new_cur.execute("SELECT title, year FROM movie ORDER BY score DESC") >>> title, year = res.fetchone() >>> print(f'The highest scoring Monty Python movie is {title!r}, released in {year}') The highest scoring Monty Python movie is 'Monty Python and the Holy Grail', released in 1975
You've now created an SQLite database using the !sqlite3
module, inserted data and retrieved values from it in multiple ways.
sqlite3-howtos
for further reading:sqlite3-placeholders
sqlite3-adapters
sqlite3-converters
sqlite3-connection-context-manager
sqlite3-explanation
for in-depth background on transaction control.
connect(database, timeout=5.0, detect_types=0, isolation_level="DEFERRED", check_same_thread=True, factory=sqlite3.Connection, cached_statements=128, uri=False)
Open a connection to an SQLite database.
- param database
The path to the database file to be opened. Pass
":memory:"
to open a connection to a database that is in RAM instead of on disk.- type database
path-like object
- param float timeout
How many seconds the connection should wait before raising an exception, if the database is locked by another connection. If another connection opens a transaction to modify the database, it will be locked until that transaction is committed. Default five seconds.
- param int detect_types
Control whether and how data types not
natively supported by SQLite <sqlite3-types>
are looked up to be converted to Python types, using the converters registered withregister_converter
. Set it to any combination (using|
, bitwise or) ofPARSE_DECLTYPES
andPARSE_COLNAMES
to enable this. Column names takes precedence over declared types if both flags are set. Types cannot be detected for generated fields (for examplemax(data)
), even when the detect_types parameter is set;str
will be returned instead. By default (0
), type detection is disabled.- param isolation_level
The
~Connection.isolation_level
of the connection, controlling whether and how transactions are implicitly opened. Can be"DEFERRED"
(default),"EXCLUSIVE"
or"IMMEDIATE"
; orNone
to disable opening transactions implicitly. Seesqlite3-controlling-transactions
for more.- type isolation_level
str | None
- param bool check_same_thread
If
True
(default), only the creating thread may use the connection. IfFalse
, the connection may be shared across multiple threads; if so, write operations should be serialized by the user to avoid data corruption.- param Connection factory
A custom subclass of
Connection
to create the connection with, if not the defaultConnection
class.- param int cached_statements
The number of statements that
!sqlite3
should internally cache for this connection, to avoid parsing overhead. By default, 128 statements.- param bool uri
If set to
True
, database is interpreted as aURI (Uniform Resource Identifier)
with a file path and an optional query string. The scheme part must be"file:"
, and the path can be relative or absolute. The query string allows passing parameters to SQLite, enabling varioussqlite3-uri-tricks
.- rtype
Connection
sqlite3.connect database sqlite3.connect
sqlite3.connect/handle connection_handle sqlite3.connect
3.4 The uri parameter.
3.7 database can now also be a path-like object
, not only a string.
3.10 The sqlite3.connect/handle
auditing event.
complete_statement(statement)
Return True
if the string statement appears to contain one or more complete SQL statements. No syntactic verification or parsing of any kind is performed, other than checking that there are no unclosed string literals and the statement is terminated by a semicolon.
For example:
>>> sqlite3.complete_statement("SELECT foo FROM bar;") True >>> sqlite3.complete_statement("SELECT foo") False
This function may be useful during command-line input to determine if the entered text seems to form a complete SQL statement, or if additional input is needed before calling ~Cursor.execute
.
enable_callback_tracebacks(flag, /)
Enable or disable callback tracebacks. By default you will not get any tracebacks in user-defined functions, aggregates, converters, authorizer callbacks etc. If you want to debug them, you can call this function with flag set to True
. Afterwards, you will get tracebacks from callbacks on sys.stderr
. Use False
to disable the feature again.
Register an unraisable hook handler <sys.unraisablehook>
for an improved debug experience:
sqlite3.trace
import sqlite3
sqlite3.trace
>>> sqlite3.enable_callback_tracebacks(True) >>> con = sqlite3.connect(":memory:") >>> def evil_trace(stmt): ... 5/0 >>> con.set_trace_callback(evil_trace) >>> def debug(unraisable): ... print(f"{unraisable.exc_value!r} in callback {unraisable.object.__name__}") ... print(f"Error message: {unraisable.err_msg}") >>> import sys >>> sys.unraisablehook = debug >>> cur = con.execute("SELECT 1") ZeroDivisionError('division by zero') in callback evil_trace Error message: None
register_adapter(type, adapter, /)
Register an adapter callable to adapt the Python type type into an SQLite type. The adapter is called with a Python object of type type as its sole argument, and must return a value of a type that SQLite natively understands <sqlite3-types>
.
register_converter(typename, converter, /)
Register the converter callable to convert SQLite objects of type typename into a Python object of a specific type. The converter is invoked for all SQLite values of type typename; it is passed a bytes
object and should return an object of the desired Python type. Consult the parameter detect_types of connect
for information regarding how type detection works.
Note: typename and the name of the type in your query are matched case-insensitively.
PARSE_COLNAMES
Pass this flag value to the detect_types parameter of connect
to look up a converter function by using the type name, parsed from the query column name, as the converter dictionary key. The type name must be wrapped in square brackets ([]
).
SELECT p as "p [point]" FROM test; ! will look up converter "point"
This flag may be combined with PARSE_DECLTYPES
using the |
(bitwise or) operator.
PARSE_DECLTYPES
Pass this flag value to the detect_types parameter of connect
to look up a converter function using the declared types for each column. The types are declared when the database table is created. !sqlite3
will look up a converter function using the first word of the declared type as the converter dictionary key. For example:
CREATE TABLE test(
i integer primary key, ! will look up a converter named "integer"
p point, ! will look up a converter named "point"
n number(10) ! will look up a converter named "number"
)
This flag may be combined with PARSE_COLNAMES
using the |
(bitwise or) operator.
SQLITE_OK SQLITE_DENY SQLITE_IGNORE
Flags that should be returned by the authorizer_callback callable passed to Connection.set_authorizer
, to indicate whether:
- Access is allowed (
!SQLITE_OK
), - The SQL statement should be aborted with an error (
!SQLITE_DENY
) - The column should be treated as a
NULL
value (!SQLITE_IGNORE
)
apilevel
String constant stating the supported DB-API level. Required by the DB-API. Hard-coded to "2.0"
.
paramstyle
String constant stating the type of parameter marker formatting expected by the !sqlite3
module. Required by the DB-API. Hard-coded to "qmark"
.
Note
The !sqlite3
module supports both qmark
and numeric
DB-API parameter styles, because that is what the underlying SQLite library supports. However, the DB-API does not allow multiple values for the paramstyle
attribute.
sqlite_version
Version number of the runtime SQLite library as a string <str>
.
sqlite_version_info
Version number of the runtime SQLite library as a tuple
of integers <int>
.
threadsafety
Integer constant required by the DB-API 2.0, stating the level of thread safety the !sqlite3
module supports. This attribute is set based on the default threading mode the underlying SQLite library is compiled with. The SQLite threading modes are:
- Single-thread: In this mode, all mutexes are disabled and SQLite is unsafe to use in more than a single thread at once.
- Multi-thread: In this mode, SQLite can be safely used by multiple threads provided that no single database connection is used simultaneously in two or more threads.
- Serialized: In serialized mode, SQLite can be safely used by multiple threads with no restriction.
The mappings from SQLite threading modes to DB-API 2.0 threadsafety levels are as follows:
SQLite threading mode | threadsafety | SQLITE_THREADSAFE | DB-API 2.0 meaning |
---|---|---|---|
single-thread | 0 | 0 | Threads may not share the module |
multi-thread | 1 | 2 | Threads may share the module, but not connections |
serialized | 3 | 1 | Threads may share the module, connections and cursors |
3.11 Set threadsafety dynamically instead of hard-coding it to 1
.
version
Version number of this module as a string <str>
. This is not the version of the SQLite library.
version_info
Version number of this module as a tuple
of integers <int>
. This is not the version of the SQLite library.
Each open SQLite database is represented by a Connection
object, which is created using sqlite3.connect
. Their main purpose is creating Cursor
objects, and sqlite3-controlling-transactions
.
sqlite3-connection-shortcuts
sqlite3-connection-context-manager
An SQLite database connection has the following attributes and methods:
isolation_level
This attribute controls the transaction handling
<sqlite3-controlling-transactions>
performed by !sqlite3
. If set to None
, transactions are never implicitly opened. If set to one of "DEFERRED"
, "IMMEDIATE"
, or "EXCLUSIVE"
, corresponding to the underlying SQLite transaction behaviour, implicit transaction management
<sqlite3-controlling-transactions>
is performed.
If not overridden by the isolation_level parameter of connect
, the default is ""
, which is an alias for "DEFERRED"
.
in_transaction
This read-only attribute corresponds to the low-level SQLite autocommit mode.
True
if a transaction is active (there are uncommitted changes), False
otherwise.
3.2
row_factory
A callable that accepts two arguments, a Cursor
object and the raw row results as a tuple
, and returns a custom object representing an SQLite row.
Example:
../includes/sqlite3/row_factory.py
If returning a tuple doesn't suffice and you want name-based access to columns, you should consider setting row_factory
to the highly optimized sqlite3.Row
type. Row
provides both index-based and case-insensitive name-based access to columns with almost no memory overhead. It will probably be better than your own custom dictionary-based approach or even a db_row based solution.
text_factory
A callable that accepts a bytes
parameter and returns a text representation of it. The callable is invoked for SQLite values with the TEXT
data type. By default, this attribute is set to str
. If you want to return bytes
instead, set text_factory to bytes
.
Example:
../includes/sqlite3/text_factory.py
total_changes
Return the total number of database rows that have been modified, inserted, or deleted since the database connection was opened.
cursor(factory=Cursor)
Create and return a Cursor
object. The cursor method accepts a single optional parameter factory. If supplied, this must be a callable returning an instance of Cursor
or its subclasses.
blobopen(table, column, row, /, *, readonly=False, name="main")
Open a Blob
handle to an existing BLOB (Binary Large OBject)
.
- param str table
The name of the table where the blob is located.
- param str column
The name of the column where the blob is located.
- param str row
The name of the row where the blob is located.
- param bool readonly
Set to
True
if the blob should be opened without write permissions. Defaults toFalse
.- param str name
The name of the database where the blob is located. Defaults to
"main"
.- raises OperationalError
When trying to open a blob in a
WITHOUT ROWID
table.- rtype
Blob
Note
The blob size cannot be changed using the Blob
class. Use the SQL function zeroblob
to create a blob with a fixed size.
3.11
commit()
Commit any pending transaction to the database. If there is no open transaction, this method is a no-op.
rollback()
Roll back to the start of any pending transaction. If there is no open transaction, this method is a no-op.
close()
Close the database connection. Any pending transaction is not committed implicitly; make sure to commit
before closing to avoid losing pending changes.
execute(sql, parameters=(), /)
Create a new Cursor
object and call ~Cursor.execute
on it with the given sql and parameters. Return the new cursor object.
executemany(sql, parameters, /)
Create a new Cursor
object and call ~Cursor.executemany
on it with the given sql and parameters. Return the new cursor object.
executescript(sql_script, /)
Create a new Cursor
object and call ~Cursor.executescript
on it with the given sql_script. Return the new cursor object.
create_function(name, narg, func, *, deterministic=False)
Create or remove a user-defined SQL function.
- param str name
The name of the SQL function.
- param int narg
The number of arguments the SQL function can accept. If
-1
, it may take any number of arguments.- param func
A callable that is called when the SQL function is invoked. The callable must return
a type natively supported by SQLite <sqlite3-types>
. Set toNone
to remove an existing SQL function.- type func
callback
| None- param bool deterministic
If
True
, the created SQL function is marked as deterministic, which allows SQLite to perform additional optimizations.- raises NotSupportedError
If deterministic is used with SQLite versions older than 3.8.3.
3.8 The deterministic parameter.
Example:
../includes/sqlite3/md5func.py
create_aggregate(name, /, n_arg, aggregate_class)
Create or remove a user-defined SQL aggregate function.
- param str name
The name of the SQL aggregate function.
- param int n_arg
The number of arguments the SQL aggregate function can accept. If
-1
, it may take any number of arguments.- param aggregate_class
A class must implement the following methods:
step()
: Add a row to the aggregate.finalize()
: Return the final result of the aggregate asa type natively supported by SQLite <sqlite3-types>
.
The number of arguments that the
step()
method must accept is controlled by n_arg.Set to
None
to remove an existing SQL aggregate function.- type aggregate_class
class
| None
Example:
../includes/sqlite3/mysumaggr.py
create_window_function(name, num_params, aggregate_class, /)
Create or remove a user-defined aggregate window function.
- param str name
The name of the SQL aggregate window function to create or remove.
- param int num_params
The number of arguments the SQL aggregate window function can accept. If
-1
, it may take any number of arguments.- param aggregate_class
A class that must implement the following methods:
step()
: Add a row to the current window.value()
: Return the current value of the aggregate.inverse()
: Remove a row from the current window.finalize()
: Return the final result of the aggregate asa type natively supported by SQLite <sqlite3-types>
.
The number of arguments that the
step()
andvalue()
methods must accept is controlled by num_params.Set to
None
to remove an existing SQL aggregate window function.- raises NotSupportedError
If used with a version of SQLite older than 3.25.0, which does not support aggregate window functions.
- type aggregate_class
class
| None
3.11
Example:
../includes/sqlite3/sumintwindow.py
create_collation(name, callable)
Create a collation named name using the collating function callable. callable is passed two string <str>
arguments, and it should return an integer <int>
:
1
if the first is ordered higher than the second-1
if the first is ordered lower than the second0
if they are ordered equal
The following example shows a reverse sorting collation:
../includes/sqlite3/collation_reverse.py
Remove a collation function by setting callable to None
.
3.11 The collation name can contain any Unicode character. Earlier, only ASCII characters were allowed.
interrupt()
Call this method from a different thread to abort any queries that might be executing on the connection. Aborted queries will raise an exception.
set_authorizer(authorizer_callback)
Register callable authorizer_callback to be invoked for each attempt to access a column of a table in the database. The callback should return one of SQLITE_OK
, SQLITE_DENY
, or SQLITE_IGNORE
to signal how access to the column should be handled by the underlying SQLite library.
The first argument to the callback signifies what kind of operation is to be authorized. The second and third argument will be arguments or None
depending on the first argument. The 4th argument is the name of the database ("main", "temp", etc.) if applicable. The 5th argument is the name of the inner-most trigger or view that is responsible for the access attempt or None
if this access attempt is directly from input SQL code.
Please consult the SQLite documentation about the possible values for the first argument and the meaning of the second and third argument depending on the first one. All necessary constants are available in the !sqlite3
module.
Passing None
as authorizer_callback will disable the authorizer.
3.11 Added support for disabling the authorizer using None
.
set_progress_handler(progress_handler, n)
Register callable progress_handler to be invoked for every n instructions of the SQLite virtual machine. This is useful if you want to get called from SQLite during long-running operations, for example to update a GUI.
If you want to clear any previously installed progress handler, call the method with None
for progress_handler.
Returning a non-zero value from the handler function will terminate the currently executing query and cause it to raise an OperationalError
exception.
set_trace_callback(trace_callback)
Register callable trace_callback to be invoked for each SQL statement that is actually executed by the SQLite backend.
The only argument passed to the callback is the statement (as str
) that is being executed. The return value of the callback is ignored. Note that the backend does not only run statements passed to the Cursor.execute
methods. Other sources include the transaction management <sqlite3-controlling-transactions>
of the !sqlite3
module and the execution of triggers defined in the current database.
Passing None
as trace_callback will disable the trace callback.
Note
Exceptions raised in the trace callback are not propagated. As a development and debugging aid, use ~sqlite3.enable_callback_tracebacks
to enable printing tracebacks from exceptions raised in the trace callback.
3.3
enable_load_extension(enabled, /)
Enable the SQLite engine to load SQLite extensions from shared libraries if enabled is True
; else, disallow loading SQLite extensions. SQLite extensions can define new functions, aggregates or whole new virtual table implementations. One well-known extension is the fulltext-search extension distributed with SQLite.
Note
The !sqlite3
module is not built with loadable extension support by default, because some platforms (notably macOS) have SQLite libraries which are compiled without this feature. To get loadable extension support, you must pass the --enable-loadable-sqlite-extensions
option to configure
.
sqlite3.enable_load_extension connection,enabled sqlite3.Connection.enable_load_extension
3.2
3.10 Added the sqlite3.enable_load_extension
auditing event.
../includes/sqlite3/load_extension.py
load_extension(path, /)
Load an SQLite extension from a shared library located at path. Enable extension loading with enable_load_extension
before calling this method.
sqlite3.load_extension connection,path sqlite3.Connection.load_extension
3.2
3.10 Added the sqlite3.load_extension
auditing event.
iterdump
Return an iterator
to dump the database as SQL source code. Useful when saving an in-memory database for later restoration. Similar to the .dump
command in the sqlite3
shell.
Example:
# Convert file example.db to SQL dump file dump.sql con = sqlite3.connect('example.db') with open('dump.sql', 'w') as f: for line in con.iterdump(): f.write('%sn' % line) con.close()
backup(target, *, pages=-1, progress=None, name="main", sleep=0.250)
Create a backup of an SQLite database.
Works even if the database is being accessed by other clients or concurrently by the same connection.
- param Connection target
The database connection to save the backup to.
- param int pages
The number of pages to copy at a time. If equal to or less than
0
, the entire database is copied in a single step. Defaults to-1
.- param progress
If set to a callable, it is invoked with three integer arguments for every backup iteration: the status of the last iteration, the remaining number of pages still to be copied, and the total number of pages. Defaults to
None
.- type progress
callback
| None- param str name
The name of the database to back up. Either
"main"
(the default) for the main database,"temp"
for the temporary database, or the name of a custom database as attached using theATTACH DATABASE
SQL statement.- param float sleep
The number of seconds to sleep between successive attempts to back up remaining pages.
Example 1, copy an existing database into another:
- def progress(status, remaining, total):
print(f'Copied {total-remaining} of {total} pages...')
src = sqlite3.connect('example.db') dst = sqlite3.connect('backup.db') with dst: src.backup(dst, pages=1, progress=progress) dst.close() src.close()
Copied 0 of 0 pages...
Example 2, copy an existing database into a transient copy:
src = sqlite3.connect('example.db') dst = sqlite3.connect(':memory:') src.backup(dst)
3.7
getlimit(category, /)
Get a connection runtime limit.
- param int category
The SQLite limit category to be queried.
- rtype
int
- raises ProgrammingError
If category is not recognised by the underlying SQLite library.
Example, query the maximum length of an SQL statement for Connection
con
(the default is 1000000000):
sqlite3.limits
import sqlite3 con = sqlite3.connect(":memory:") con.setlimit(sqlite3.SQLITE_LIMIT_SQL_LENGTH, 1_000_000_000) con.setlimit(sqlite3.SQLITE_LIMIT_ATTACHED, 10)
sqlite3.limits
>>> con.getlimit(sqlite3.SQLITE_LIMIT_SQL_LENGTH) 1000000000
3.11
setlimit(category, limit, /)
Set a connection runtime limit. Attempts to increase a limit above its hard upper bound are silently truncated to the hard upper bound. Regardless of whether or not the limit was changed, the prior value of the limit is returned.
- param int category
The SQLite limit category to be set.
- param int limit
The value of the new limit. If negative, the current limit is unchanged.
- rtype
int
- raises ProgrammingError
If category is not recognised by the underlying SQLite library.
Example, limit the number of attached databases to 1 for Connection
con
(the default limit is 10):
sqlite3.limits
>>> con.setlimit(sqlite3.SQLITE_LIMIT_ATTACHED, 1) 10 >>> con.getlimit(sqlite3.SQLITE_LIMIT_ATTACHED) 1
3.11
serialize(*, name="main")
Serialize a database into a bytes
object. For an ordinary on-disk database file, the serialization is just a copy of the disk file. For an in-memory database or a "temp" database, the serialization is the same sequence of bytes which would be written to disk if that database were backed up to disk.
- param str name
The database name to be serialized. Defaults to
"main"
.- rtype
bytes
Note
This method is only available if the underlying SQLite library has the serialize API.
3.11
deserialize(data, /, *, name="main")
Deserialize a serialized <serialize>
database into a Connection
. This method causes the database connection to disconnect from database name, and reopen name as an in-memory database based on the serialization contained in data.
- param bytes data
A serialized database.
- param str name
The database name to deserialize into. Defaults to
"main"
.- raises OperationalError
If the database connection is currently involved in a read transaction or a backup operation.
- raises DatabaseError
If data does not contain a valid SQLite database.
- raises OverflowError
If
len(data) <len>
is larger than2**63 - 1
.
Note
This method is only available if the underlying SQLite library has the deserialize API.
3.11
A
Cursor
object represents a database cursor which is used to execute SQL statements, and manage the context of a fetch operation. Cursors are created usingConnection.cursor
, or by using any of theconnection shortcut methods <sqlite3-connection-shortcuts>
.Cursor objects are
iterators <iterator>
, meaning that if you~Cursor.execute
aSELECT
query, you can simply iterate over the cursor to fetch the resulting rows:sqlite3.cursor
import sqlite3 con = sqlite3.connect(":memory:", isolation_level=None) cur = con.execute("CREATE TABLE data(t)") cur.execute("INSERT INTO data VALUES(1)")
sqlite3.cursor
- for row in cur.execute("SELECT t FROM data"):
print(row)
sqlite3.cursor
(1,)
A Cursor
instance has the following attributes and methods.
single: ? (question mark); in SQL statements
single: : (colon); in SQL statements
execute(sql, parameters=(), /)
Execute SQL statement sql. Bind values to the statement using placeholders
<sqlite3-placeholders>
that map to the sequence
or dict
parameters.
execute
will only execute a single SQL statement. If you try to execute more than one statement with it, it will raise a ProgrammingError
. Use executescript
if you want to execute multiple SQL statements with one call.
If ~Connection.isolation_level
is not None
, sql is an INSERT
, UPDATE
, DELETE
, or REPLACE
statement, and there is no open transaction, a transaction is implicitly opened before executing sql.
executemany(sql, parameters, /)
Execute parameterized <sqlite3-placeholders>
SQL statement sql against all parameter sequences or mappings found in the sequence parameters. It is also possible to use an iterator
yielding parameters instead of a sequence. Uses the same implicit transaction handling as ~Cursor.execute
.
Example:
sqlite3.cursor
- rows = [
("row1",), ("row2",),
] # cur is an sqlite3.Cursor object cur.executemany("INSERT INTO data VALUES(?)", rows)
executescript(sql_script, /)
Execute the SQL statements in sql_script. If there is a pending transaction, an implicit COMMIT
statement is executed first. No other implicit transaction control is performed; any transaction control must be added to sql_script.
sql_script must be a string <str>
.
Example:
sqlite3.cursor
# cur is an sqlite3.Cursor object cur.executescript(""" BEGIN; CREATE TABLE person(firstname, lastname, age); CREATE TABLE book(title, author, published); CREATE TABLE publisher(name, address); COMMIT; """)
fetchone()
Return the next row of a query result set as a tuple
. Return None
if no more data is available.
fetchmany(size=cursor.arraysize)
Return the next set of rows of a query result as a list
. Return an empty list if no more rows are available.
The number of rows to fetch per call is specified by the size parameter. If size is not given, arraysize
determines the number of rows to be fetched. If fewer than size rows are available, as many rows as are available are returned.
Note there are performance considerations involved with the size parameter. For optimal performance, it is usually best to use the arraysize attribute. If the size parameter is used, then it is best for it to retain the same value from one fetchmany
call to the next.
fetchall()
Return all (remaining) rows of a query result as a list
. Return an empty list if no rows are available. Note that the arraysize
attribute can affect the performance of this operation.
close()
Close the cursor now (rather than whenever __del__
is called).
The cursor will be unusable from this point forward; a ProgrammingError
exception will be raised if any operation is attempted with the cursor.
setinputsizes(sizes, /)
Required by the DB-API. Does nothing in !sqlite3
.
setoutputsize(size, column=None, /)
Required by the DB-API. Does nothing in !sqlite3
.
rowcount
Read-only attribute that provides the number of modified rows for INSERT
, UPDATE
, DELETE
, and REPLACE
statements; is -1
for other statements, including CTE (Common Table Expression)
queries. It is only updated by the execute
and executemany
methods.
lastrowid
Read-only attribute that provides the row id of the last inserted row. It is only updated after successful INSERT
or REPLACE
statements using the execute
method. For other statements, after executemany
or executescript
, or if the insertion failed, the value of lastrowid
is left unchanged. The initial value of lastrowid
is None
.
Note
Inserts into WITHOUT ROWID
tables are not recorded.
3.6 Added support for the REPLACE
statement.
arraysize
Read/write attribute that controls the number of rows returned by fetchmany
. The default value is 1 which means a single row would be fetched per call.
description
Read-only attribute that provides the column names of the last query. To remain compatible with the Python DB API, it returns a 7-tuple for each column where the last six items of each tuple are None
.
It is set for SELECT
statements without any matching rows as well.
connection
Read-only attribute that provides the SQLite database Connection
belonging to the cursor. A Cursor
object created by calling con.cursor() <Connection.cursor>
will have a connection
attribute that refers to con:
>>> con = sqlite3.connect(":memory:") >>> cur = con.cursor() >>> cur.connection == con True
A !Row
instance serves as a highly optimized ~Connection.row_factory
for Connection
objects. It supports iteration, equality testing, len
, and mapping
access by column name and index.
Two row objects compare equal if have equal columns and equal members.
keys
Return a list
of column names as strings <str>
. Immediately after a query, it is the first member of each tuple in Cursor.description
.
3.5 Added support of slicing.
Example:
>>> con = sqlite3.connect(":memory:") >>> con.row_factory = sqlite3.Row >>> res = con.execute("SELECT 'Earth' AS name, 6378 AS radius") >>> row = res.fetchone() >>> row.keys() ['name', 'radius'] >>> row[0], row["name"] # Access by index and name. ('Earth', 'Earth') >>> row["RADIUS"] # Column names are case-insensitive. 6378
3.11
A Blob
instance is a file-like object
that can read and write data in an SQLite BLOB (Binary Large OBject)
. Call len(blob) <len>
to get the size (number of bytes) of the blob. Use indices and slices <slice>
for direct access to the blob data.
Use the Blob
as a context manager
to ensure that the blob handle is closed after use.
../includes/sqlite3/blob.py
close()
Close the blob.
The blob will be unusable from this point onward. An ~sqlite3.Error
(or subclass) exception will be raised if any further operation is attempted with the blob.
read(length=-1, /)
Read length bytes of data from the blob at the current offset position. If the end of the blob is reached, the data up to EOF (End of File)
will be returned. When length is not specified, or is negative, ~Blob.read
will read until the end of the blob.
write(data, /)
Write data to the blob at the current offset. This function cannot change the blob length. Writing beyond the end of the blob will raise ValueError
.
tell()
Return the current access position of the blob.
seek(offset, origin=os.SEEK_SET, /)
Set the current access position of the blob to offset. The origin argument defaults to os.SEEK_SET
(absolute blob positioning). Other values for origin are os.SEEK_CUR
(seek relative to the current position) and os.SEEK_END
(seek relative to the blob’s end).
The PrepareProtocol type's single purpose is to act as a 246
style adaption protocol for objects that can adapt themselves
<sqlite3-conform>
to native SQLite types <sqlite3-types>
.
The exception hierarchy is defined by the DB-API 2.0 (249
).
Warning
This exception is not currently raised by the !sqlite3
module, but may be raised by applications using !sqlite3
, for example if a user-defined function truncates data while inserting. Warning
is a subclass of Exception
.
Error
The base class of the other exceptions in this module. Use this to catch all errors with one single except
statement. Error
is a subclass of Exception
.
If the exception originated from within the SQLite library, the following two attributes are added to the exception:
InterfaceError
Exception raised for misuse of the low-level SQLite C API. In other words, if this exception is raised, it probably indicates a bug in the !sqlite3
module. InterfaceError
is a subclass of Error
.
DatabaseError
Exception raised for errors that are related to the database. This serves as the base exception for several types of database errors. It is only raised implicitly through the specialised subclasses. DatabaseError
is a subclass of Error
.
DataError
Exception raised for errors caused by problems with the processed data, like numeric values out of range, and strings which are too long. DataError
is a subclass of DatabaseError
.
OperationalError
Exception raised for errors that are related to the database's operation, and not necessarily under the control of the programmer. For example, the database path is not found, or a transaction could not be processed. OperationalError
is a subclass of DatabaseError
.
IntegrityError
Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails. It is a subclass of DatabaseError
.
InternalError
Exception raised when SQLite encounters an internal error. If this is raised, it may indicate that there is a problem with the runtime SQLite library. InternalError
is a subclass of DatabaseError
.
ProgrammingError
Exception raised for !sqlite3
API programming errors, for example supplying the wrong number of bindings to a query, or trying to operate on a closed Connection
. ProgrammingError
is a subclass of DatabaseError
.
NotSupportedError
Exception raised in case a method or database API is not supported by the underlying SQLite library. For example, setting deterministic to True
in ~Connection.create_function
, if the underlying SQLite library does not support deterministic functions. NotSupportedError
is a subclass of DatabaseError
.
SQLite natively supports the following types: NULL
, INTEGER
, REAL
, TEXT
, BLOB
.
The following Python types can thus be sent to SQLite without any problem:
Python type | SQLite type |
---|---|
None |
NULL |
int |
INTEGER |
float |
REAL |
str |
TEXT |
bytes |
BLOB |
This is how SQLite types are converted to Python types by default:
SQLite type | Python type |
---|---|
NULL |
None |
INTEGER |
int |
REAL |
float |
TEXT |
depends on ~Connection.text_factory , str by default |
BLOB |
bytes |
The type system of the !sqlite3
module is extensible in two ways: you can store additional Python types in an SQLite database via object adapters <sqlite3-adapters>
, and you can let the !sqlite3
module convert SQLite types to Python types via converters <sqlite3-converters>
.
There are default adapters for the date and datetime types in the datetime module. They will be sent as ISO dates/ISO timestamps to SQLite.
The default converters are registered under the name "date" for datetime.date
and under the name "timestamp" for datetime.datetime
.
This way, you can use date/timestamps from Python without any additional fiddling in most cases. The format of the adapters is also compatible with the experimental SQLite date/time functions.
The following example demonstrates this.
../includes/sqlite3/pysqlite_datetime.py
If a timestamp stored in SQLite has a fractional part longer than 6 numbers, its value will be truncated to microsecond precision by the timestamp converter.
Note
The default "timestamp" converter ignores UTC offsets in the database and always returns a naive datetime.datetime
object. To preserve UTC offsets in timestamps, either leave converters disabled, or register an offset-aware converter with register_converter
.
SQL operations usually need to use values from Python variables. However, beware of using Python's string operations to assemble queries, as they are vulnerable to SQL injection attacks (see the xkcd webcomic for a humorous example of what can go wrong):
# Never do this -- insecure!
symbol = 'RHAT'
cur.execute("SELECT * FROM stocks WHERE symbol = '%s'" % symbol)
Instead, use the DB-API's parameter substitution. To insert a variable into a query string, use a placeholder in the string, and substitute the actual values into the query by providing them as a tuple
of values to the second argument of the cursor's ~Cursor.execute
method. An SQL statement may use one of two kinds of placeholders: question marks (qmark style) or named placeholders (named style). For the qmark style, parameters
must be a sequence <sequence>
. For the named style, it can be either a sequence <sequence>
or dict
instance. The length of the sequence <sequence>
must match the number of placeholders, or a ProgrammingError
is raised. If a dict
is given, it must contain keys for all named parameters. Any extra items are ignored. Here's an example of both styles:
../includes/sqlite3/execute_1.py
SQLite supports only a limited set of data types natively. To store custom Python types in SQLite databases, adapt them to one of the Python types SQLite natively understands <sqlite3-types>
.
There are two ways to adapt Python objects to SQLite types: letting your object adapt itself, or using an adapter callable. The latter will take precedence above the former. For a library that exports a custom type, it may make sense to enable that type to adapt itself. As an application developer, it may make more sense to take direct control by registering custom adapter functions.
Suppose we have a !Point
class that represents a pair of coordinates, x
and y
, in a Cartesian coordinate system. The coordinate pair will be stored as a text string in the database, using a semicolon to separate the coordinates. This can be implemented by adding a __conform__(self, protocol)
method which returns the adapted value. The object passed to protocol will be of type PrepareProtocol
.
../includes/sqlite3/adapter_point_1.py
The other possibility is to create a function that converts the Python object to an SQLite-compatible type. This function can then be registered using register_adapter
.
../includes/sqlite3/adapter_point_2.py
Writing an adapter lets you convert from custom Python types to SQLite values. To be able to convert from SQLite values to custom Python types, we use converters.
Let's go back to the !Point
class. We stored the x and y coordinates separated via semicolons as strings in SQLite.
First, we'll define a converter function that accepts the string as a parameter and constructs a !Point
object from it.
Note
Converter functions are always passed a bytes
object, no matter the underlying SQLite data type.
- def convert_point(s):
x, y = map(float, s.split(b";")) return Point(x, y)
We now need to tell !sqlite3
when it should convert a given SQLite value. This is done when connecting to a database, using the detect_types parameter of connect
. There are three options:
- Implicit: set detect_types to
PARSE_DECLTYPES
- Explicit: set detect_types to
PARSE_COLNAMES
- Both: set detect_types to
sqlite3.PARSE_DECLTYPES | sqlite3.PARSE_COLNAMES
. Column names take precedence over declared types.
The following example illustrates the implicit and explicit approaches:
../includes/sqlite3/converter_point.py
This section shows recipes for common adapters and converters.
import datetime import sqlite3
- def adapt_date_iso(val):
"""Adapt datetime.date to ISO 8601 date.""" return val.isoformat()
- def adapt_datetime_iso(val):
"""Adapt datetime.datetime to timezone-naive ISO 8601 date.""" return val.isoformat()
- def adapt_datetime_epoch(val):
"""Adapt datetime.datetime to Unix timestamp.""" return int(val.timestamp())
sqlite3.register_adapter(datetime.date, adapt_date_iso) sqlite3.register_adapter(datetime.datetime, adapt_datetime_iso) sqlite3.register_adapter(datetime.datetime, adapt_datetime_epoch)
- def convert_date(val):
"""Convert ISO 8601 date to datetime.date object.""" return datetime.date.fromisoformat(val)
- def convert_datetime(val):
"""Convert ISO 8601 datetime to datetime.datetime object.""" return datetime.datetime.fromisoformat(val)
- def convert_timestamp(val):
"""Convert Unix epoch timestamp to datetime.datetime object.""" return datetime.datetime.fromtimestamp(val)
sqlite3.register_converter("date", convert_date) sqlite3.register_converter("datetime", convert_datetime) sqlite3.register_converter("timestamp", convert_timestamp)
Using the ~Connection.execute
, ~Connection.executemany
, and ~Connection.executescript
methods of the Connection
class, your code can be written more concisely because you don't have to create the (often superfluous) Cursor
objects explicitly. Instead, the Cursor
objects are created implicitly and these shortcut methods return the cursor objects. This way, you can execute a SELECT
statement and iterate over it directly using only a single call on the Connection
object.
../includes/sqlite3/shortcut_methods.py
A Connection
object can be used as a context manager that automatically commits or rolls back open transactions when leaving the body of the context manager. If the body of the with
statement finishes without exceptions, the transaction is committed. If this commit fails, or if the body of the with
statement raises an uncaught exception, the transaction is rolled back.
If there is no open transaction upon leaving the body of the with
statement, the context manager is a no-op.
Note
The context manager neither implicitly opens a new transaction nor closes the connection.
../includes/sqlite3/ctx_manager.py
Some useful URI tricks include:
- Open a database in read-only mode:
>>> con = sqlite3.connect("file:tutorial.db?mode=ro", uri=True) >>> con.execute("CREATE TABLE readonly(data)") Traceback (most recent call last): OperationalError: attempt to write a readonly database
- Do not implicitly create a new database file if it does not already exist; will raise
~sqlite3.OperationalError
if unable to create a new file:
>>> con = sqlite3.connect("file:nosuchdb.db?mode=rw", uri=True) Traceback (most recent call last): OperationalError: unable to open database file
- Create a shared named in-memory database:
db = "file:mem1?mode=memory&cache=shared" con1 = sqlite3.connect(db, uri=True) con2 = sqlite3.connect(db, uri=True) with con1: con1.execute("CREATE TABLE shared(data)") con1.execute("INSERT INTO shared VALUES(28)") res = con2.execute("SELECT data FROM shared") assert res.fetchone() == (28,)
More information about this feature, including a list of parameters, can be found in the SQLite URI documentation.
The !sqlite3
module does not adhere to the transaction handling recommended by 249
.
If the connection attribute ~Connection.isolation_level
is not None
, new transactions are implicitly opened before ~Cursor.execute
and ~Cursor.executemany
executes INSERT
, UPDATE
, DELETE
, or REPLACE
statements. Use the ~Connection.commit
and ~Connection.rollback
methods to respectively commit and roll back pending transactions. You can choose the underlying SQLite transaction behaviour — that is, whether and what type of BEGIN
statements !sqlite3
implicitly executes – via the ~Connection.isolation_level
attribute.
If ~Connection.isolation_level
is set to None
, no transactions are implicitly opened at all. This leaves the underlying SQLite library in autocommit mode, but also allows the user to perform their own transaction handling using explicit SQL statements. The underlying SQLite library autocommit mode can be queried using the ~Connection.in_transaction
attribute.
The ~Cursor.executescript
method implicitly commits any pending transaction before execution of the given SQL script, regardless of the value of ~Connection.isolation_level
.
3.6 !sqlite3
used to implicitly commit an open transaction before DDL statements. This is no longer the case.