Skip to content

Commit

Permalink
Add more STRICT table support (#604)
Browse files Browse the repository at this point in the history
* Add more STRICT table support per #344 (comment).
* Make `table.transform()` preserve STRICT mode.
* Fix mypy failures in PR #604
* Link to SQLITE strict page in a few places
  • Loading branch information
tkhattra committed Dec 8, 2023
1 parent 88bd372 commit 1500c19
Show file tree
Hide file tree
Showing 9 changed files with 182 additions and 5 deletions.
3 changes: 3 additions & 0 deletions docs/cli-reference.rst
Expand Up @@ -289,6 +289,7 @@ See :ref:`cli_inserting_data`, :ref:`cli_insert_csv_tsv`, :ref:`cli_insert_unstr
--analyze Run ANALYZE at the end of this operation
--load-extension TEXT Path to SQLite extension, with optional :entrypoint
--silent Do not show progress bar
--strict Apply STRICT mode to created table
--ignore Ignore records if pk already exists
--replace Replace records if pk already exists
--truncate Truncate table before inserting records, if table
Expand Down Expand Up @@ -345,6 +346,7 @@ See :ref:`cli_upsert`.
--analyze Run ANALYZE at the end of this operation
--load-extension TEXT Path to SQLite extension, with optional :entrypoint
--silent Do not show progress bar
--strict Apply STRICT mode to created table
-h, --help Show this message and exit.


Expand Down Expand Up @@ -920,6 +922,7 @@ See :ref:`cli_create_table`.
--replace If table already exists, replace it
--transform If table already exists, try to transform the schema
--load-extension TEXT Path to SQLite extension, with optional :entrypoint
--strict Apply STRICT mode to created table
-h, --help Show this message and exit.


Expand Down
21 changes: 20 additions & 1 deletion docs/cli.rst
Expand Up @@ -1972,6 +1972,25 @@ You can specify foreign key relationships between the tables you are creating us
[author_id] INTEGER REFERENCES [authors]([id])
)
You can create a table in `SQLite STRICT mode <https://www.sqlite.org/stricttables.html>`__ using ``--strict``:

.. code-block:: bash
sqlite-utils create-table mydb.db mytable id integer name text --strict
.. code-block:: bash
sqlite-utils tables mydb.db --schema -t
.. code-block:: output
table schema
------- ------------------------
mytable CREATE TABLE [mytable] (
[id] INTEGER,
[name] TEXT
) STRICT
If a table with the same name already exists, you will get an error. You can choose to silently ignore this error with ``--ignore``, or you can replace the existing table with a new, empty table using ``--replace``.

You can also pass ``--transform`` to transform the existing table to match the new schema. See :ref:`python_api_explicit_create` in the Python library documentation for details of how this option works.
Expand Down Expand Up @@ -2018,7 +2037,7 @@ Use ``--ignore`` to ignore the error if the table does not exist.
Transforming tables
===================

The ``transform`` command allows you to apply complex transformations to a table that cannot be implemented using a regular SQLite ``ALTER TABLE`` command. See :ref:`python_api_transform` for details of how this works.
The ``transform`` command allows you to apply complex transformations to a table that cannot be implemented using a regular SQLite ``ALTER TABLE`` command. See :ref:`python_api_transform` for details of how this works. The ``transform`` command preserves a table's ``STRICT`` mode.

.. code-block:: bash
Expand Down
18 changes: 17 additions & 1 deletion docs/python-api.rst
Expand Up @@ -117,6 +117,12 @@ By default, any :ref:`sqlite-utils plugins <plugins>` that implement the :ref:`p
db = Database(memory=True, execute_plugins=False)
You can pass ``strict=True`` to enable `SQLite STRICT mode <https://www.sqlite.org/stricttables.html>`__ for all tables created using this database object:

.. code-block:: python
db = Database("my_database.db", strict=True)
.. _python_api_attach:

Attaching additional databases
Expand Down Expand Up @@ -581,6 +587,15 @@ The ``transform=True`` option will update the table schema if any of the followi

Changes to ``foreign_keys=`` are not currently detected and applied by ``transform=True``.

You can pass ``strict=True`` to create a table in ``STRICT`` mode:

.. code-block:: python
db["cats"].create({
"id": int,
"name": str,
}, strict=True)
.. _python_api_compound_primary_keys:

Compound primary keys
Expand Down Expand Up @@ -661,7 +676,7 @@ You can set default values for these methods by accessing the table through the
# Now you can call .insert() like so:
table.insert({"id": 1, "name": "Tracy", "score": 5})
The configuration options that can be specified in this way are ``pk``, ``foreign_keys``, ``column_order``, ``not_null``, ``defaults``, ``batch_size``, ``hash_id``, ``hash_id_columns``, ``alter``, ``ignore``, ``replace``, ``extracts``, ``conversions``, ``columns``. These are all documented below.
The configuration options that can be specified in this way are ``pk``, ``foreign_keys``, ``column_order``, ``not_null``, ``defaults``, ``batch_size``, ``hash_id``, ``hash_id_columns``, ``alter``, ``ignore``, ``replace``, ``extracts``, ``conversions``, ``columns``, ``strict``. These are all documented below.

.. _python_api_defaults_not_null:

Expand Down Expand Up @@ -1011,6 +1026,7 @@ The first time this is called the record will be created for ``name="Palm"``. An
- ``extracts``
- ``conversions``
- ``columns``
- ``strict``

.. _python_api_extracts:

Expand Down
19 changes: 19 additions & 0 deletions sqlite_utils/cli.py
Expand Up @@ -909,6 +909,12 @@ def inner(fn):
),
load_extension_option,
click.option("--silent", is_flag=True, help="Do not show progress bar"),
click.option(
"--strict",
is_flag=True,
default=False,
help="Apply STRICT mode to created table",
),
)
):
fn = decorator(fn)
Expand Down Expand Up @@ -951,6 +957,7 @@ def insert_upsert_implementation(
silent=False,
bulk_sql=None,
functions=None,
strict=False,
):
db = sqlite_utils.Database(path)
_load_extensions(db, load_extension)
Expand Down Expand Up @@ -1066,6 +1073,7 @@ def insert_upsert_implementation(
"replace": replace,
"truncate": truncate,
"analyze": analyze,
"strict": strict,
}
if not_null:
extra_kwargs["not_null"] = set(not_null)
Expand Down Expand Up @@ -1186,6 +1194,7 @@ def insert(
truncate,
not_null,
default,
strict,
):
"""
Insert records from FILE into a table, creating the table if it
Expand Down Expand Up @@ -1264,6 +1273,7 @@ def insert(
silent=silent,
not_null=not_null,
default=default,
strict=strict,
)
except UnicodeDecodeError as ex:
raise click.ClickException(UNICODE_ERROR.format(ex))
Expand Down Expand Up @@ -1299,6 +1309,7 @@ def upsert(
analyze,
load_extension,
silent,
strict,
):
"""
Upsert records based on their primary key. Works like 'insert' but if
Expand Down Expand Up @@ -1343,6 +1354,7 @@ def upsert(
analyze=analyze,
load_extension=load_extension,
silent=silent,
strict=strict,
)
except UnicodeDecodeError as ex:
raise click.ClickException(UNICODE_ERROR.format(ex))
Expand Down Expand Up @@ -1511,6 +1523,11 @@ def create_database(path, enable_wal, init_spatialite, load_extension):
help="If table already exists, try to transform the schema",
)
@load_extension_option
@click.option(
"--strict",
is_flag=True,
help="Apply STRICT mode to created table",
)
def create_table(
path,
table,
Expand All @@ -1523,6 +1540,7 @@ def create_table(
replace,
transform,
load_extension,
strict,
):
"""
Add a table with the specified columns. Columns should be specified using
Expand Down Expand Up @@ -1570,6 +1588,7 @@ def create_table(
ignore=ignore,
replace=replace,
transform=transform,
strict=strict,
)


Expand Down

0 comments on commit 1500c19

Please sign in to comment.