diff --git a/docs/_extra/robots.txt b/docs/_extra/robots.txt index 412dae65..baa43f3c 100644 --- a/docs/_extra/robots.txt +++ b/docs/_extra/robots.txt @@ -1,4 +1,2 @@ -User-agent: * -Disallow: / - Sitemap: https://crate.io/docs/python/en/latest/site.xml +User-agent: * diff --git a/docs/appendices/data-types.rst b/docs/appendices/data-types.rst index b0d15e16..d6a34e3b 100644 --- a/docs/appendices/data-types.rst +++ b/docs/appendices/data-types.rst @@ -156,6 +156,3 @@ __ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#o __ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#array __ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-point __ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-shape - -.. _json: https://docs.python.org/3/library/json.html -.. _HTTP endpoint: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html diff --git a/docs/blobs.rst b/docs/blobs.rst index 6249330c..365865eb 100644 --- a/docs/blobs.rst +++ b/docs/blobs.rst @@ -4,8 +4,9 @@ Blobs ===== -The CrateDB Python client library provides full access to the powerful `blob -storage capabilities`_ of your CrateDB cluster. +The CrateDB Python client library provides full access to the powerful +:ref:`blob storage capabilities ` of your +CrateDB cluster. .. rubric:: Table of contents @@ -25,12 +26,14 @@ For the sake of this example, we will do the following: >>> connection = client.connect("http://localhost:4200/") This is a simple connection that connects to a CrateDB node running on -the local host with the `HTTP endpoint`_ listening on port 4200 (the default). +the local host with the :ref:`crate-reference:interface-http` listening +on port 4200 (the default). -To work with blobs in CrateDB, you must specifically `create blob tables`_. +To work with blobs in CrateDB, you must specifically create +:ref:`blob tables `. The CrateDB Python client allows you to interact with these blob tables via a -blob container, which you can create like this:: +blob container, which you can create like this: >>> blob_container = connection.get_blob_container('my_blobs') >>> blob_container @@ -53,31 +56,31 @@ produce bytes when read. What is a file-like object? Well, to put it simply, any object that provides a ``read()`` method. -The stream objects provided by the Python standard library `io`_ module and -`tempfile`_ module are the most commonly used file-like objects. +The stream objects provided by the Python standard library :mod:`py:io` and +:mod:`py:tempfile` modules are the most commonly used file-like objects. -The `StringIO`_ class is not suitable, as it produces Unicode strings when -read. But you can easily encode a Unicode string and feed it to a `BytesIO`_ +The :class:`py:io.StringIO` class is not suitable, as it produces Unicode strings when +read. But you can easily encode a Unicode string and feed it to a :class:`py:io.BytesIO` object. -Here's a trivial example:: +Here's a trivial example: >>> import io >>> bytestream = "An example sentence.".encode("utf8") >>> file = io.BytesIO(bytestream) -This file can then be uploaded to the blob table using the ``put`` method:: +This file can then be uploaded to the blob table using the ``put`` method: >>> blob_container.put(file) '6f10281ad07d4a35c6ec2f993e6376032b77181d' -Notice that this method computes and returns a `SHA-1 digest`_. This is +Notice that this method computes and returns an `SHA-1 digest`_. This is necessary for attempting to save the blob to CrateDB. If you already have the SHA-1 digest computed, or are able to compute it as part of an existing read, this may improve the performance of your application. -If you pass in a SHA-1 digest, it will not be recomputed:: +If you pass in a SHA-1 digest, it will not be recomputed: >>> file.seek(0) # seek to the beginning before attempting to re-upload @@ -123,12 +126,12 @@ You can get the blob, with the ``get`` method, like so: >>> blob_generator = blob_container.get(digest) Blobs are read in chunks. The default size of these chunks is 128 kilobytes, -but this can be changed by suppling the desired chunk size to the ``get`` -method, like so:: +but this can be changed by supplying the desired chunk size to the ``get`` +method, like so: >>> res = blob_container.get(digest, 1024 * 128) -The ``blob`` object is a Python `generator`_, meaning that you can call +The ``blob`` object is a Python :term:`py:generator`, meaning that you can call ``next(blob)`` for each new chunk you want to read, until you encounter a ``StopIteration`` exception. @@ -143,7 +146,7 @@ generator is like so: Delete blobs ------------ -You can delete a blob with the ``delete`` method and the blob digest, like so:: +You can delete a blob with the ``delete`` method and the blob digest, like so: >>> blob_container.delete(digest) True @@ -156,12 +159,4 @@ We can verify that, like so: >>> blob_container.exists(digest) False -.. _blob storage capabilities: https://crate.io/docs/crate/reference/en/latest/general/blobs.html -.. _BytesIO: https://docs.python.org/3/library/io.html#binary-i-o -.. _create blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html#creating-a-table-for-blobs -.. _generator: https://stackoverflow.com/a/1756156 -.. _HTTP endpoint: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html -.. _io: https://docs.python.org/3/library/io.html -.. _SHA-1 digest: https://docs.python.org/3/library/hashlib.html -.. _StringIO: https://docs.python.org/3/library/io.html#io.StringIO -.. _tempfile: https://docs.python.org/3/library/tempfile.html +.. _SHA-1 digest: https://en.wikipedia.org/wiki/SHA-1 diff --git a/docs/by-example/blob.rst b/docs/by-example/blob.rst index 589f566f..3b5f6ca3 100644 --- a/docs/by-example/blob.rst +++ b/docs/by-example/blob.rst @@ -2,19 +2,19 @@ Blob container API ================== -The connection object provides a convenience API for easy access to `blob -tables`_. +The connection object provides a convenience API for easy access to +:ref:`blob tables `. Get blob container handle ========================= -Create a connection:: +Create a connection: >>> from crate.client import connect >>> client = connect([crate_host]) -Get a blob container:: +Get a blob container: >>> container = client.get_blob_container('myfiles') @@ -26,7 +26,7 @@ The container allows to store a blob without explicitly providing the hash for the blob. This feature is possible if the blob is provided as a seekable stream like object. -Store a ``StringIO`` stream:: +Store a ``StringIO`` stream: >>> from io import BytesIO >>> f = BytesIO(b'StringIO data') @@ -34,7 +34,7 @@ Store a ``StringIO`` stream:: >>> stringio_bob '0cd4511d696823779692484029f234471cd21f28' -Store from a file:: +Store from a file: >>> from tempfile import TemporaryFile >>> f = TemporaryFile() @@ -46,7 +46,7 @@ Store from a file:: >>> f.close() If the blob data is not provided as a seekable stream the hash must be -provided explicitly:: +provided explicitly: >>> import hashlib >>> string_data = b'String data' @@ -67,7 +67,7 @@ Check for existence Retrieve blobs ============== -Blobs can be retrieved using its hash:: +Blobs can be retrieved using its hash: >>> blob_stream = container.get(string_blob) >>> blob_stream @@ -80,14 +80,14 @@ Blobs can be retrieved using its hash:: Delete blobs ============ -Blobs can be deleted using its hash:: +Blobs can be deleted using its hash: >>> container.delete(string_blob) True >>> container.exists(string_blob) False -Trying to delete a not existing blob:: +Trying to delete a not existing blob: >>> container.delete(string_blob) False @@ -95,9 +95,6 @@ Trying to delete a not existing blob:: Close connection ================ -Close the connection to clear the connection pool:: +Close the connection to clear the connection pool: >>> client.close() - - -.. _blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html diff --git a/docs/by-example/client.rst b/docs/by-example/client.rst index 047bfc12..c9046d68 100644 --- a/docs/by-example/client.rst +++ b/docs/by-example/client.rst @@ -16,12 +16,12 @@ and closing the connection again. Connect to a database ===================== -Before we can start we have to import the client:: +Before we can start we have to import the client: >>> from crate import client The client provides a ``connect()`` function which is used to establish a -connection, the first argument is the url of the server to connect to:: +connection, the first argument is the url of the server to connect to: >>> connection = client.connect(crate_host) >>> connection.close() @@ -29,13 +29,13 @@ connection, the first argument is the url of the server to connect to:: CrateDB is a clustered database providing high availability through replication. In order for clients to make use of this property it is recommended to specify all hosts of the cluster. This way if a server does not -respond, the request is automatically routed to the next server:: +respond, the request is automatically routed to the next server: >>> invalid_host = 'http://not_responding_host:4200' >>> connection = client.connect([invalid_host, crate_host]) >>> connection.close() -If no ``servers`` are given, the default one ``http://127.0.0.1:4200`` is used:: +If no ``servers`` are given, the default one ``http://127.0.0.1:4200`` is used: >>> connection = client.connect() >>> connection.client._active_servers @@ -43,13 +43,13 @@ If no ``servers`` are given, the default one ``http://127.0.0.1:4200`` is used:: >>> connection.close() If the option ``error_trace`` is set to ``True``, the client will print a whole -traceback if a server error occurs:: +traceback if a server error occurs: >>> connection = client.connect([crate_host], error_trace=True) >>> connection.close() It's possible to define a default timeout value in seconds for all servers -using the optional parameter ``timeout``:: +using the optional parameter ``timeout``: >>> connection = client.connect([crate_host, invalid_host], timeout=5) >>> connection.close() @@ -59,7 +59,7 @@ Authentication Users that are trusted as by definition of the ``auth.host_based.config`` setting do not need a password, but only require the ``username`` argument to -connect:: +connect: >>> connection = client.connect([crate_host], ... username='trusted_me') @@ -68,7 +68,7 @@ connect:: >>> connection.client.password >>> connection.close() -The username for trusted users can also be provided in the URL:: +The username for trusted users can also be provided in the URL: >>> connection = client.connect(['http://trusted_me@' + crate_host]) >>> connection.client.username @@ -77,7 +77,7 @@ The username for trusted users can also be provided in the URL:: >>> connection.close() To connect to CrateDB with as a user that requires password authentication, you -also need to provide ``password`` as argument for the ``connect()`` call:: +also need to provide ``password`` as argument for the ``connect()`` call: >>> connection = client.connect([crate_host], ... username='me', @@ -88,7 +88,7 @@ also need to provide ``password`` as argument for the ``connect()`` call:: 'my_secret_pw' >>> connection.close() -The authentication credentials can also be provided in the URL:: +The authentication credentials can also be provided in the URL: >>> connection = client.connect(['http://me:my_secret_pw@' + crate_host]) >>> connection.client.username @@ -102,7 +102,7 @@ Default Schema -------------- To connect to CrateDB and use a different default schema than ``doc``, you can -provide the ``schema`` keyword argument in the ``connect()`` method, like so:: +provide the ``schema`` keyword argument in the ``connect()`` method, like so: >>> connection = client.connect([crate_host], ... schema='custom_schema') @@ -111,19 +111,19 @@ provide the ``schema`` keyword argument in the ``connect()`` method, like so:: Inserting Data ============== -Use user "crate" for rest of the tests:: +Use user "crate" for rest of the tests: >>> connection = client.connect([crate_host]) Before executing any statement, a cursor has to be opened to perform -database operations:: +database operations: >>> cursor = connection.cursor() >>> cursor.execute("""INSERT INTO locations ... (name, date, kind, position) VALUES (?, ?, ?, ?)""", ... ('Einstein Cross', '2007-03-11', 'Quasar', 7)) -To bulk insert data you can use the ``executemany`` function:: +To bulk insert data you can use the ``executemany`` function: >>> cursor.executemany("""INSERT INTO locations ... (name, date, kind, position) VALUES (?, ?, ?, ?)""", @@ -143,7 +143,7 @@ Selecting Data ============== To perform the select operation simply execute the statement on the -open cursor:: +open cursor: >>> cursor.execute("SELECT name FROM locations where name = ?", ('Algol',)) @@ -152,13 +152,13 @@ To retrieve a row we can use one of the cursor's fetch functions (described belo fetchone() ---------- -``fetchone()`` with each call returns the next row from the results:: +``fetchone()`` with each call returns the next row from the results: >>> result = cursor.fetchone() >>> pprint(result) ['Algol'] -If no more data is available, an empty result is returned:: +If no more data is available, an empty result is returned: >>> while cursor.fetchone(): ... pass @@ -168,7 +168,7 @@ fetchmany() ----------- ``fetch_many()`` returns a list of all remaining rows, containing no more than -the specified size of rows:: +the specified size of rows: >>> cursor.execute("SELECT name FROM locations order by name") >>> result = cursor.fetchmany(2) @@ -176,12 +176,12 @@ the specified size of rows:: [['Aldebaran'], ['Algol']] If a size is not given, the cursor's arraysize, which defaults to '1', -determines the number of rows to be fetched:: +determines the number of rows to be fetched: >>> cursor.fetchmany() [['Allosimanius Syneca']] -It's also possible to change the cursors arraysize to an other value:: +It's also possible to change the cursors arraysize to an other value: >>> cursor.arraysize = 3 >>> cursor.fetchmany() @@ -190,7 +190,7 @@ It's also possible to change the cursors arraysize to an other value:: fetchall() ---------- -``fetchall()`` returns a list of all remaining rows:: +``fetchall()`` returns a list of all remaining rows: >>> cursor.execute("SELECT name FROM locations order by name") >>> result = cursor.fetchall() @@ -217,7 +217,7 @@ Cursor Description The ``description`` property of the cursor returns a sequence of 7-item sequences containing the column name as first parameter. Just the name field is -supported, all other fields are 'None':: +supported, all other fields are 'None': >>> cursor.execute("SELECT * FROM locations order by name") >>> result = cursor.fetchone() @@ -253,7 +253,7 @@ supported, all other fields are 'None':: Closing the Cursor ================== -The following command closes the cursor:: +The following command closes the cursor: >>> cursor.close() @@ -270,13 +270,13 @@ be raised. Closing the Connection ====================== -The following command closes the connection:: +The following command closes the connection: >>> connection.close() If a connection is closed, it will be unusable from this point forward. If any operation using the connection is attempted to a closed connection an -``ProgrammingError`` will be raised:: +``ProgrammingError`` will be raised: >>> cursor.execute("SELECT * FROM locations") Traceback (most recent call last): diff --git a/docs/by-example/connection.rst b/docs/by-example/connection.rst index 6aafb1f2..4b89db7d 100644 --- a/docs/by-example/connection.rst +++ b/docs/by-example/connection.rst @@ -31,12 +31,12 @@ cursor() ======== Calling the ``cursor()`` function on the connection will -return a cursor object:: +return a cursor object: >>> cursor = connection.cursor() Now we are able to perform any operation provided by the -cursor object:: +cursor object: >>> cursor.rowcount -1 @@ -44,13 +44,13 @@ cursor object:: close() ======= -Now we close the connection:: +Now we close the connection: >>> connection.close() The connection will be unusable from this point. Any operation attempted with the closed connection will -raise a ``ProgrammingError``:: +raise a ``ProgrammingError``: >>> cursor = connection.cursor() Traceback (most recent call last): diff --git a/docs/by-example/cursor.rst b/docs/by-example/cursor.rst index 0b979f4d..7fc7da7d 100644 --- a/docs/by-example/cursor.rst +++ b/docs/by-example/cursor.rst @@ -50,30 +50,30 @@ request without needing to execute an SQL statement. fetchone() ========== -Calling ``fetchone()`` on the cursor object the first time after an execute returns the first row:: +Calling ``fetchone()`` on the cursor object the first time after an execute returns the first row: >>> cursor.execute('') >>> cursor.fetchone() ['North West Ripple', 1] -Each call to ``fetchone()`` increments the cursor and returns the next row:: +Each call to ``fetchone()`` increments the cursor and returns the next row: >>> cursor.fetchone() ['Arkintoofle Minor', 3] -One more iteration:: +One more iteration: >>> cursor.next() ['Alpha Centauri', 3] The iteration is stopped after the last row is returned. -A further call to ``fetchone()`` returns an empty result:: +A further call to ``fetchone()`` returns an empty result: >>> cursor.fetchone() Using ``fetchone()`` on a cursor before issuing a database statement results -in an error:: +in an error: >>> new_cursor = connection.cursor() >>> new_cursor.fetchone() @@ -85,21 +85,21 @@ in an error:: fetchmany() =========== -``fetchmany()`` takes an argument which specifies the number of rows we want to fetch:: +``fetchmany()`` takes an argument which specifies the number of rows we want to fetch: >>> cursor.execute('') >>> cursor.fetchmany(2) [['North West Ripple', 1], ['Arkintoofle Minor', 3]] -If the specified number of rows not being available, fewer rows may returned:: +If the specified number of rows not being available, fewer rows may returned: >>> cursor.fetchmany(2) [['Alpha Centauri', 3]] >>> cursor.execute('') -If no number of rows are specified it defaults to the current ``cursor.arraysize``:: +If no number of rows are specified it defaults to the current ``cursor.arraysize``: >>> cursor.arraysize 1 @@ -112,7 +112,7 @@ If no number of rows are specified it defaults to the current ``cursor.arraysize >>> cursor.fetchmany() [['North West Ripple', 1], ['Arkintoofle Minor', 3]] -If zero number of rows are specified, all rows left are returned:: +If zero number of rows are specified, all rows left are returned: >>> cursor.fetchmany(0) [['Alpha Centauri', 3]] @@ -120,18 +120,18 @@ If zero number of rows are specified, all rows left are returned:: fetchall() ========== -``fetchall()`` fetches all (remaining) rows of a query result:: +``fetchall()`` fetches all (remaining) rows of a query result: >>> cursor.execute('') >>> cursor.fetchall() [['North West Ripple', 1], ['Arkintoofle Minor', 3], ['Alpha Centauri', 3]] -Since all data was fetched 'None' is returned by ``fetchone()``:: +Since all data was fetched 'None' is returned by ``fetchone()``: >>> cursor.fetchone() -And each other call returns an empty sequence:: +And each other call returns an empty sequence: >>> cursor.fetchmany(2) [] @@ -145,14 +145,14 @@ And each other call returns an empty sequence:: iteration ========= -The cursor supports the iterator interface and can be iterated upon:: +The cursor supports the iterator interface and can be iterated upon: >>> cursor.execute('') >>> [row for row in cursor] [['North West Ripple', 1], ['Arkintoofle Minor', 3], ['Alpha Centauri', 3]] When no other call to execute has been done, it will raise StopIteration on -subsequent iterations:: +subsequent iterations: >>> next(cursor) Traceback (most recent call last): @@ -166,7 +166,7 @@ subsequent iterations:: ['Arkintoofle Minor', 3] ['Alpha Centauri', 3] -Iterating over a new cursor without results will immediately raise a ProgrammingError:: +Iterating over a new cursor without results will immediately raise a ProgrammingError: >>> new_cursor = connection.cursor() >>> next(new_cursor) @@ -183,19 +183,19 @@ description rowcount ======== -The ``rowcount`` property specifies the number of rows that the last ``execute()`` produced:: +The ``rowcount`` property specifies the number of rows that the last ``execute()`` produced: >>> cursor.execute('') >>> cursor.rowcount 3 -The attribute is -1 in case the cursor has been closed:: +The attribute is ``-1``, in case the cursor has been closed: >>> cursor.close() >>> cursor.rowcount -1 -If the last response does not contain the rowcount attribute, ``-1`` is returned:: +If the last response does not contain the rowcount attribute, ``-1`` is returned: >>> cursor = connection.cursor() >>> connection.client.set_next_response({ @@ -219,14 +219,14 @@ duration ======== The ``duration`` property specifies the server-side duration in milliseconds of the last query -issued by ``execute()``:: +issued by ``execute()``: >>> cursor = connection.cursor() >>> cursor.execute('') >>> cursor.duration 123 -The attribute is -1 in case the cursor has been closed:: +The attribute is ``-1``, in case the cursor has been closed: >>> cursor.close() >>> cursor.duration @@ -249,7 +249,7 @@ executemany() ============= ``executemany()`` allows to execute a single sql statement against a sequence -of parameters:: +of parameters: >>> cursor = connection.cursor() @@ -262,12 +262,12 @@ of parameters:: 123 ``executemany()`` is not intended to be used with statements returning result -sets. The result will always be empty:: +sets. The result will always be empty: >>> cursor.fetchall() [] -For completeness' sake the cursor description is updated nonetheless:: +For completeness' sake the cursor description is updated nonetheless: >>> [ desc[0] for desc in cursor.description ] ['name', 'position'] @@ -284,7 +284,7 @@ close() ======= After closing a cursor the connection will be unusable. If any operation is attempted with the -closed connection an ``ProgrammingError`` exception will be raised:: +closed connection an ``ProgrammingError`` exception will be raised: >>> cursor = connection.cursor() >>> cursor.execute('') @@ -320,8 +320,6 @@ The cursor object can optionally convert database types to native Python data types. Currently, this is implemented for the CrateDB data types ``IP`` and ``TIMESTAMP`` on behalf of the ``DefaultTypeConverter``. -:: - >>> cursor = connection.cursor(converter=DefaultTypeConverter()) >>> connection.client.set_next_response({ @@ -344,7 +342,8 @@ Custom data type conversion By providing a custom converter instance, you can define your own data type conversions. For investigating the list of available data types, please either inspect the ``DataType`` enum, or the documentation about the list of available -`CrateDB data type identifiers for the HTTP interface`_. +:ref:`CrateDB data type identifiers for the HTTP interface +`. To create a simple converter for converging CrateDB's ``BIT`` type to Python's ``int`` type. @@ -442,6 +441,3 @@ Let's exercise all of them: .. Hidden: close connection >>> connection.close() - - -.. _CrateDB data type identifiers for the HTTP interface: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html#column-types diff --git a/docs/by-example/http.rst b/docs/by-example/http.rst index 0a36458a..494e7b65 100644 --- a/docs/by-example/http.rst +++ b/docs/by-example/http.rst @@ -20,25 +20,25 @@ The CrateDB Python driver package offers an HTTP client API object. Server configuration ==================== -A list of servers can be passed while creating an instance of the http client:: +A list of servers can be passed while creating an instance of the http client: >>> http_client = HttpClient([crate_host]) >>> http_client.close() -Its also possible to pass a single server as a string:: +Its also possible to pass a single server as a string: >>> http_client = HttpClient(crate_host) >>> http_client.close() If no ``server`` argument (or no argument at all) is passed, the default one -``127.0.0.1:4200`` is used:: +``127.0.0.1:4200`` is used: >>> http_client = HttpClient() >>> http_client._active_servers ['http://127.0.0.1:4200'] >>> http_client.close() -When using a list of servers, the servers are selected by round-robin:: +When using a list of servers, the servers are selected by round-robin: >>> invalid_host = "invalid_host:9999" >>> even_more_invalid_host = "even_more_invalid_host:9999" @@ -54,7 +54,7 @@ When using a list of servers, the servers are selected by round-robin:: >>> http_client.close() -Servers with connection errors will be removed from the active server list:: +Servers with connection errors will be removed from the active server list: >>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host]) >>> result = http_client.sql('select name from locations') @@ -62,7 +62,7 @@ Servers with connection errors will be removed from the active server list:: ['http://127.0.0.1:44209'] Inactive servers will be re-added after a given time interval. -To validate this, set the interval very short and sleep for that interval:: +To validate this, set the interval very short and sleep for that interval: >>> http_client.retry_interval = 1 >>> import time; time.sleep(1) @@ -74,7 +74,7 @@ To validate this, set the interval very short and sleep for that interval:: >>> http_client.close() If no active servers are available and the retry interval is not reached, just use the oldest -inactive one:: +inactive one: >>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host]) >>> result = http_client.sql('select name from locations') @@ -86,7 +86,7 @@ inactive one:: SQL Statements ============== -Issue a select statement against our with test data pre-filled crate instance:: +Issue a select statement against our with test data pre-filled crate instance: >>> http_client = HttpClient(crate_host) >>> result = http_client.sql('select name from locations order by name') @@ -112,19 +112,19 @@ Issue a select statement against our with test data pre-filled crate instance:: Blobs ===== -Check if a blob exists:: +Check if a blob exists: >>> http_client.blob_exists('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8') False -Trying to get a non-existing blob throws an exception:: +Trying to get a non-existing blob throws an exception: >>> http_client.blob_get('myfiles', '041f06fd774092478d450774f5ba30c5da78acc8') Traceback (most recent call last): ... crate.client.exceptions.DigestNotFoundException: myfiles/041f06fd774092478d450774f5ba30c5da78acc8 -Creating a new blob - this method returns ``True`` if the blob was newly created:: +Creating a new blob - this method returns ``True`` if the blob was newly created: >>> from tempfile import TemporaryFile >>> f = TemporaryFile() @@ -134,25 +134,25 @@ Creating a new blob - this method returns ``True`` if the blob was newly created ... 'myfiles', '040f06fd774092478d450774f5ba30c5da78acc8', f) True -Uploading the same content again returns ``False``:: +Uploading the same content again returns ``False``: >>> _ = f.seek(0) >>> http_client.blob_put( ... 'myfiles', '040f06fd774092478d450774f5ba30c5da78acc8', f) False -Now the blob exist:: +Now the blob exist: >>> http_client.blob_exists('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8') True -Blobs are returned as generators, generating a chunk on each call:: +Blobs are returned as generators, generating a chunk on each call: >>> g = http_client.blob_get('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8') >>> print(next(g)) content -The chunk_size can be set explicitly on get:: +The chunk_size can be set explicitly on get: >>> g = http_client.blob_get( ... 'myfiles', '040f06fd774092478d450774f5ba30c5da78acc8', 5) @@ -162,7 +162,7 @@ The chunk_size can be set explicitly on get:: >>> print(next(g)) nt -Deleting a blob - this method returns true if the blob existed:: +Deleting a blob - this method returns true if the blob existed: >>> http_client.blob_del('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8') True @@ -170,7 +170,7 @@ Deleting a blob - this method returns true if the blob existed:: >>> http_client.blob_del('myfiles', '040f06fd774092478d450774f5ba30c5da78acc8') False -Uploading a blob to a table with disabled blob support throws an exception:: +Uploading a blob to a table with disabled blob support throws an exception: >>> _ = f.seek(0) >>> http_client.blob_put( @@ -187,7 +187,7 @@ Error Handling ============== Create a function that takes a lot of time to return so we can run into a -timeout exception:: +timeout exception: >>> http_client = HttpClient(crate_host) >>> http_client.sql(''' @@ -200,7 +200,7 @@ timeout exception:: >>> http_client.close() It's possible to define a HTTP timeout in seconds on client instantiation, so -an exception is raised when the timeout is reached:: +an exception is raised when the timeout is reached: >>> http_client = HttpClient(crate_host, timeout=0.01) >>> http_client.sql('select fib(32)') @@ -209,7 +209,7 @@ an exception is raised when the timeout is reached:: crate.client.exceptions.ConnectionError: No more Servers available, exception from last server: ... >>> http_client.close() -When connecting to non-CrateDB servers, the HttpClient will raise a ConnectionError like this:: +When connecting to non-CrateDB servers, the HttpClient will raise a ConnectionError like this: >>> http_client = HttpClient(["https://example.org/"]) >>> http_client.server_infos(http_client._get_server()) @@ -220,7 +220,7 @@ When connecting to non-CrateDB servers, the HttpClient will raise a ConnectionEr >>> http_client.close() When using the ``error_trace`` kwarg a full traceback of the server exception -will be provided:: +will be provided: >>> from crate.client.exceptions import ProgrammingError >>> http_client = HttpClient([crate_host], error_trace=True) diff --git a/docs/by-example/https.rst b/docs/by-example/https.rst index 8dadf301..cc6da50b 100644 --- a/docs/by-example/https.rst +++ b/docs/by-example/https.rst @@ -40,13 +40,13 @@ The CrateDB Python driver package offers a HTTP client API object. With certificate verification ============================= -When using a valid CA certificate, the connection will be successful:: +When using a valid CA certificate, the connection will be successful: >>> client = HttpClient([crate_host], ca_cert=cacert_valid) >>> client.server_infos(client._get_server()) ('https://localhost:65534', 'test', '0.0.0') -When not providing a ``ca_cert`` file, the connection will fail:: +When not providing a ``ca_cert`` file, the connection will fail: >>> client = HttpClient([crate_host]) >>> client.server_infos(crate_host) @@ -54,7 +54,7 @@ When not providing a ``ca_cert`` file, the connection will fail:: ... crate.client.exceptions.ConnectionError: Server not available, ...certificate verify failed... -Also, when providing an invalid ``ca_cert``, an error is raised:: +Also, when providing an invalid ``ca_cert``, an error is raised: >>> client = HttpClient([crate_host], ca_cert=cacert_invalid) >>> client.server_infos(crate_host) @@ -67,14 +67,14 @@ Without certificate verification ================================ When turning off certificate verification, calling the server will succeed, -even when not providing a valid CA certificate:: +even when not providing a valid CA certificate: >>> client = HttpClient([crate_host], verify_ssl_cert=False) >>> client.server_infos(crate_host) ('https://localhost:65534', 'test', '0.0.0') Without verification, calling the server will even work when using an invalid -``ca_cert``:: +``ca_cert``: >>> client = HttpClient([crate_host], verify_ssl_cert=False, ca_cert=cacert_invalid) >>> client.server_infos(crate_host) @@ -89,13 +89,13 @@ The CrateDB driver also supports client certificates. The ``HttpClient`` constructor takes two keyword arguments: ``cert_file`` and ``key_file``. Both should be strings pointing to the path of the client -certificate and key file:: +certificate and key file: >>> client = HttpClient([crate_host], ca_cert=cacert_valid, cert_file=clientcert_valid, key_file=clientcert_valid) >>> client.server_infos(crate_host) ('https://localhost:65534', 'test', '0.0.0') -When using an invalid client certificate, the connection will fail:: +When using an invalid client certificate, the connection will fail: >>> client = HttpClient([crate_host], ca_cert=cacert_valid, cert_file=clientcert_invalid, key_file=clientcert_invalid) >>> client.server_infos(crate_host) @@ -103,7 +103,7 @@ When using an invalid client certificate, the connection will fail:: ... crate.client.exceptions.ConnectionError: Server not available, exception: HTTPSConnectionPool... -The connection will also fail when providing an invalid CA certificate:: +The connection will also fail when providing an invalid CA certificate: >>> client = HttpClient([crate_host], ca_cert=cacert_invalid, cert_file=clientcert_valid, key_file=clientcert_valid) >>> client.server_infos(crate_host) diff --git a/docs/by-example/index.rst b/docs/by-example/index.rst index 044225e5..dcb9be4c 100644 --- a/docs/by-example/index.rst +++ b/docs/by-example/index.rst @@ -16,8 +16,8 @@ DBAPI, HTTP, and BLOB interfaces The examples in this section are all about CrateDB's `Python DBAPI`_ interface, the plain HTTP API interface, and a convenience interface for working with -`blob tables`_. It details attributes, methods, and behaviors of the -``Connection`` and ``Cursor`` objects. +:ref:`blob tables `. It details attributes, +methods, and behaviors of the ``Connection`` and ``Cursor`` objects. .. toctree:: :maxdepth: 1 @@ -32,8 +32,8 @@ the plain HTTP API interface, and a convenience interface for working with .. _sqlalchemy-by-example: -SQLAlchemy interface -==================== +SQLAlchemy by example +===================== The examples in this section are all about CrateDB's `SQLAlchemy`_ dialect, and its corresponding API interfaces, see also :ref:`sqlalchemy-support`. @@ -48,6 +48,5 @@ its corresponding API interfaces, see also :ref:`sqlalchemy-support`. sqlalchemy/inspection-reflection -.. _blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html .. _Python DBAPI: https://peps.python.org/pep-0249/ .. _SQLAlchemy: https://www.sqlalchemy.org/ diff --git a/docs/by-example/sqlalchemy/advanced-querying.rst b/docs/by-example/sqlalchemy/advanced-querying.rst index ff62511c..026ef635 100644 --- a/docs/by-example/sqlalchemy/advanced-querying.rst +++ b/docs/by-example/sqlalchemy/advanced-querying.rst @@ -58,7 +58,7 @@ to conduct fulltext search queries, we need to create a table with a We have to create this table using SQL because it is currently not possible to create ``INDEX`` fields using SQLAlchemy's :ref:`sa:orm_declarative_mapping`. -But we can define the table to use all other operations: +However, we can define the table to use all other operations: >>> def gen_key(): ... return str(uuid4()) @@ -242,7 +242,7 @@ Now, let's use ``insert().from_select()`` to archive the task into the >>> result = session.execute(ins) >>> session.commit() -This will emit the following ``INSERT`` statement to the database:: +This will emit the following ``INSERT`` statement to the database: INSERT INTO archived_tasks (id, content) (SELECT todos.id, todos.content FROM todos WHERE todos.status = 'done') diff --git a/docs/by-example/sqlalchemy/crud.rst b/docs/by-example/sqlalchemy/crud.rst index ba56b271..a84404f3 100644 --- a/docs/by-example/sqlalchemy/crud.rst +++ b/docs/by-example/sqlalchemy/crud.rst @@ -5,13 +5,13 @@ SQLAlchemy: Create, retrieve, update, and delete ================================================ This section of the documentation shows how to query, insert, update and delete -data using CrateDB's SQLAlchemy integration, it includes common scenarios like: - +records using CrateDB's SQLAlchemy integration, it includes common scenarios +like: - Filtering records - Limiting result sets - Inserts and updates with default values -- Updating complex data types with nested dictionaries + .. rubric:: Table of Contents @@ -31,12 +31,14 @@ Import the relevant symbols: >>> from sqlalchemy.orm import sessionmaker >>> from crate.client.sqlalchemy.types import ObjectArray -Establish a connection to the database: +Establish a connection to the database, see also :ref:`sa:engines_toplevel` +and :ref:`connect`: >>> engine = sa.create_engine(f"crate://{crate_host}") >>> connection = engine.connect() -Define the ORM schema for the ``Location`` entity: +Define the ORM schema for the ``Location`` entity using SQLAlchemy's +:ref:`sa:orm_declarative_mapping`: >>> Base = declarative_base(bind=engine) @@ -52,47 +54,10 @@ Define the ORM schema for the ``Location`` entity: ... flag = sa.Column(sa.Boolean) ... details = sa.Column(ObjectArray) -Create a session with SQLAlchemy: +Create an SQLAlchemy :doc:`Session `: >>> session = sessionmaker(bind=engine)() -Retrieve -======== - -Using the connection to execute a select statement: - - >>> result = connection.execute('select name from locations order by name') - >>> result.rowcount - 13 - - >>> result.first() - ('Aldebaran',) - -Using the ORM to query the locations: - - >>> locations = session.query(Location).order_by('name') - >>> [l.name for l in locations if l is not None][:2] - ['Aldebaran', 'Algol'] - -With limit and offset: - - >>> locations = session.query(Location).order_by('name').offset(1).limit(2) - >>> [l.name for l in locations if l is not None] - ['Algol', 'Allosimanius Syneca'] - -With filter: - - >>> location = session.query(Location).filter_by(name='Algol').one() - >>> location.name - 'Algol' - -Order by: - - >>> locations = session.query(Location).filter(Location.name is not None).order_by(sa.desc(Location.name)) - >>> locations = locations.limit(2) - >>> [l.name for l in locations] - ['Outer Eastern Rim', 'North West Ripple'] - Create ====== @@ -157,9 +122,51 @@ them, they are not set when the record is inserted: True +Retrieve +======== + +Using the connection to execute a select statement: + + >>> result = connection.execute('select name from locations order by name') + >>> result.rowcount + 14 + + >>> result.first() + ('Aldebaran',) + +Using the ORM to query the locations: + + >>> locations = session.query(Location).order_by('name') + >>> [l.name for l in locations if l is not None][:2] + ['Aldebaran', 'Algol'] + +With limit and offset: + + >>> locations = session.query(Location).order_by('name').offset(1).limit(2) + >>> [l.name for l in locations if l is not None] + ['Algol', 'Allosimanius Syneca'] + +With filter: + + >>> location = session.query(Location).filter_by(name='Algol').one() + >>> location.name + 'Algol' + +Order by: + + >>> locations = session.query(Location).filter(Location.name is not None).order_by(sa.desc(Location.name)) + >>> locations = locations.limit(2) + >>> [l.name for l in locations] + ['Outer Eastern Rim', 'North West Ripple'] + + Update ====== +Back to our original object ``Location(Earth)``. + + >>> location = session.query(Location).filter_by(name='Earth').one() + The datetime and date can be set using an update statement: >>> location.nullable_date = datetime.utcnow().date() @@ -206,14 +213,15 @@ Refresh table: >>> _ = connection.execute("REFRESH TABLE locations") -Query database: +Update multiple records using SQL: >>> result = connection.execute("update locations set flag=true where kind='Update'") >>> result.rowcount 10 -Check that number of affected documents of update without ``where-clause`` matches number of all -documents in the table: +Update all records using SQL, and check that the number of documents affected +of an update without ``where-clause`` matches the number of all documents in +the table: >>> result = connection.execute(u"update locations set kind='Überall'") >>> result.rowcount == connection.execute("select * from locations limit 100").rowcount diff --git a/docs/by-example/sqlalchemy/inspection-reflection.rst b/docs/by-example/sqlalchemy/inspection-reflection.rst index 45bd0263..1d811a17 100644 --- a/docs/by-example/sqlalchemy/inspection-reflection.rst +++ b/docs/by-example/sqlalchemy/inspection-reflection.rst @@ -11,28 +11,35 @@ SQLAlchemy integration. Introduction ============ +The CrateDB SQLAlchemy integration provides different ways to inspect the +database. -The CrateDB SQLAlchemy integration provides different ways to inspect the database. +1) The :ref:`runtime inspection API ` allows you to get + an ``Inspector`` instance that can be used to fetch schema names, table names + and other information. -1) The `runtime inspection API`_ allows you to get an ``Inspector`` instance that can be used to fetch schema names, table names and other information. +2) Reflection capabilities allow you to create ``Table`` instances from + existing tables to inspect their columns and constraints. -2) Reflection capabilities allow you to create ``Table`` instances from existing tables to inspect their columns and constraints. - -3) A ``CrateDialect`` allows you to get connection information and it contains low level function to check the existence of schemas and tables. +3) A ``CrateDialect`` allows you to get connection information and it contains + low level function to check the existence of schemas and tables. All approaches require an ``Engine`` instance, which you can create like this: >>> import sqlalchemy as sa >>> engine = sa.create_engine(f"crate://{crate_host}") +This effectively establishes a connection to the database, see also +:ref:`sa:engines_toplevel` and :ref:`connect`. + Inspector ========= -The `SQLAlchemy inspector`_ is a low level interface which provides a -backend-agnostic system of loading lists of schema, table, column, and -constraint descriptions from a given database. You create an inspector like -this: +The :ref:`SQLAlchemy inspector ` is a low +level interface which provides a backend-agnostic system of loading lists of +schema, table, column, and constraint descriptions from a given database. +You can create an inspector like this: >>> inspector = sa.inspect(engine) @@ -64,8 +71,8 @@ Schema-supported reflection =========================== A ``Table`` object can load its own schema information from the corresponding -table in the database. This process is called *reflection*, see `reflecting -database objects`_. +table in the database. This process is called *reflection*, see +:ref:`sa:metadata_reflection`. In the most simple case you need only specify the table name, a ``MetaData`` object, and the ``autoload_with`` argument. @@ -118,8 +125,3 @@ Check if a table exists: >>> connection.close() >>> engine.dispose() - - -.. _reflecting database objects: https://docs.sqlalchemy.org/en/14/core/reflection.html#reflecting-database-objects -.. _runtime inspection API: https://docs.sqlalchemy.org/en/14/core/inspection.html -.. _SQLAlchemy inspector: https://docs.sqlalchemy.org/en/14/core/reflection.html#fine-grained-reflection-with-inspector diff --git a/docs/by-example/sqlalchemy/working-with-types.rst b/docs/by-example/sqlalchemy/working-with-types.rst index 52af180d..1016c439 100644 --- a/docs/by-example/sqlalchemy/working-with-types.rst +++ b/docs/by-example/sqlalchemy/working-with-types.rst @@ -211,7 +211,7 @@ Geospatial types CrateDB's geospatial types, such as :ref:`crate-reference:type-geo_point` and :ref:`crate-reference:type-geo_shape`, can also be used within an -SQLAlchemy declarative schema. +SQLAlchemy declarative schema: >>> class City(Base): ... __tablename__ = 'cities' diff --git a/docs/conf.py b/docs/conf.py index 23b61e90..8267b131 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -10,10 +10,15 @@ intersphinx_mapping.update({ + 'py': ('https://docs.python.org/3/', None), 'sa': ('https://docs.sqlalchemy.org/en/14/', None), + 'urllib3': ('https://urllib3.readthedocs.io/en/1.26.13/', None) }) +linkcheck_anchors = True + + rst_prolog = """ .. |nbsp| unicode:: 0xA0 :trim: diff --git a/docs/connect.rst b/docs/connect.rst index a60c7b37..cd0a7481 100644 --- a/docs/connect.rst +++ b/docs/connect.rst @@ -6,42 +6,41 @@ Connect to CrateDB .. NOTE:: - This page documents the CrateDB `Database API`_ client. + This page documents the CrateDB client, implementing the + `Python Database API Specification v2.0`_ (PEP 249). - For help using the `SQLAlchemy`_ dialect, consult - :ref:`the SQLAlchemy dialect documentation `. + For help using the `SQLAlchemy`_ dialect, consult the + :ref:`SQLAlchemy dialect documentation `. .. SEEALSO:: - Supplementary information about the CrateDB Database API client can be found - in the :ref:`data types appendix `. - - For general help using the Database API, consult `PEP 0249`_. + Supplementary information about the CrateDB Database API client can be found + in the :ref:`data types appendix `. .. rubric:: Table of contents .. contents:: - :local: + :local: .. _single-node: Connect to a single node ======================== -To connect to a single CrateDB node, use the ``connect()`` function, like so:: +To connect to a single CrateDB node, use the ``connect()`` function, like so: >>> connection = client.connect("", username="") -Here, replace ```` with a URL pointing to the `HTTP endpoint`_ of a -CrateDB node. Replace ```` with the username you are authenticating -as. +Here, replace ```` with a URL pointing to the +:ref:`crate-reference:interface-http` of a CrateDB node. Replace ```` +with the username you are authenticating as. .. NOTE:: - This example authenticates as ``crate``, the default database user in - CrateDB versions 2.1.x and later. This might not work for you. + This example authenticates as ``crate``, which is the default + :ref:`database user `. - Consult the `Authentication`_ section for more information. + Consult the `Authentication`_ section for more information. Example node URLs: @@ -55,12 +54,12 @@ for HTTP requests on port 4200, the node URL would be .. TIP:: - If a ```` argument is not provided, the library will attempt - to connect to CrateDB on the local host with the default HTTP port number, - i.e. ``http://localhost:4200/``. + If a ```` argument is not provided, the library will attempt + to connect to CrateDB on the local host with the default HTTP port number, + i.e. ``http://localhost:4200/``. - If you're just getting started with CrateDB, the first time you connect, - you can probably omit this argument. + If you're just getting started with CrateDB, the first time you connect, + you can probably omit this argument. .. _multiple-nodes: @@ -68,7 +67,7 @@ Connect to multiple nodes ========================= To connect to one of multiple nodes, pass a list of database URLs to the -connect() function, like so:: +connect() function, like so: >>> connection = client.connect(["", ""], ...) @@ -101,22 +100,23 @@ URL: .. SEEALSO:: - The CrateDB reference has a section on `setting up SSL`_. This will be - a useful background reading for the following two subsections. + The CrateDB reference has a section on :ref:`setting up SSL + `. This will be a useful background reading for + the following two subsections. Server verification ................... Server certificates are verified by default. In order to connect to a SSL-enabled host using self-signed certificates, you will need to provide the -CA certificate file used to sign the server SSL certificate:: +CA certificate file used to sign the server SSL certificate: >>> connection = client.connect(..., ca_cert="") Here, replace ```` with the path to the CA certificate file. You can disable server SSL certificate verification by using the -``verify_ssl_cert`` keyword argument and setting it to ``False``:: +``verify_ssl_cert`` keyword argument and setting it to ``False``: >>> connection = client.connect(..., verify_ssl_cert=False) @@ -126,7 +126,7 @@ Client verification The client also supports client verification via client certificates. -Here's how you might do that:: +Here's how you might do that: >>> connection = client.connect(..., cert_file="", key_file="") @@ -143,7 +143,7 @@ Timeout ------- Connection timeouts (in seconds) can be configured with the optional -``timeout`` argument:: +``timeout`` argument: >>> connection = client.connect(..., timeout=5) @@ -151,14 +151,14 @@ Here, replace ``...`` with the rest of your arguments. .. NOTE:: - If no timeout is specified, the client will use the default Python `socket - timeout`_. + If no timeout is specified, the client will use the default Python + :func:`socket timeout `. Tracebacks ---------- -`Tracebacks`_ in the event of a connection error will be printed if you set -the optional ``error_trace`` argument to ``True``, like so:: +In the event of a connection error, a :mod:`py:traceback` will be printed, if +you set the optional ``error_trace`` argument to ``True``, like so: >>> connection = client.connect(..., error_trace=True) @@ -166,7 +166,7 @@ Backoff Factor -------------- When attempting to make a request, the connection can be configured so that -retries are made in increasing time intervals. This can be configured like so:: +retries are made in increasing time intervals. This can be configured like so: >>> connection = client.connect(..., backoff_factor=0.1) @@ -177,10 +177,11 @@ default its value is 0. Socket Options -------------- -Creating connections uses `urllib3 default socket options`_ but additionally -enables TCP keepalive by setting ``socket.SO_KEEPALIVE`` to ``1``. +Creating connections uses :class:`urllib3 default socket options +` but additionally enables TCP +keepalive by setting ``socket.SO_KEEPALIVE`` to ``1``. -Keepalive can be disabled using the ``socket_keepalive`` argument, like so:: +Keepalive can be disabled using the ``socket_keepalive`` argument, like so: >>> connection = client.connect(..., socket_keepalive=False) @@ -217,11 +218,11 @@ Authentication If you are using CrateDB 2.1.x or later, you must supply a username. If you are using earlier versions of CrateDB, this argument is not supported. -You can authenticate with CrateDB like so:: +You can authenticate with CrateDB like so: >>> connection = client.connect(..., username="", password="") -At your disposal, you can also embed the credentials into the URI, like so:: +At your disposal, you can also embed the credentials into the URI, like so: >>> connection = client.connect("https://:@cratedb.example.org:4200") @@ -230,16 +231,17 @@ and password. .. TIP:: - If you have not configured a custom `database user`_, you probably want to - authenticate as the CrateDB superuser, which is ``crate``. The superuser - does not have a password, so you can omit the ``password`` argument. + If you have not configured a custom :ref:`database user + `, you probably want to + authenticate as the CrateDB superuser, which is ``crate``. The superuser + does not have a password, so you can omit the ``password`` argument. .. _schema-selection: Schema selection ================ -You can select a schema using the optional ``schema`` argument, like so:: +You can select a schema using the optional ``schema`` argument, like so: >>> connection = client.connect(..., schema="") @@ -260,19 +262,14 @@ Once you're connected, you can :ref:`query CrateDB `. .. SEEALSO:: - Check out the `sample application`_ (and the corresponding `documentation`_) - for a practical demonstration of this driver in use. + Check out the `sample application`_ (and the corresponding `sample + application documentation`_) for a practical demonstration of this driver + in use. + .. _client-side random load balancing: https://en.wikipedia.org/wiki/Load_balancing_(computing)#Client-side_random_load_balancing -.. _Database API: http://www.python.org/dev/peps/pep-0249/ -.. _database user: https://crate.io/docs/crate/reference/en/latest/admin/user-management.html -.. _documentation: https://github.com/crate/crate-sample-apps/blob/master/python/documentation.md -.. _HTTP endpoint: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html -.. _PEP 0249: http://www.python.org/dev/peps/pep-0249/ +.. _Python Database API Specification v2.0: https://www.python.org/dev/peps/pep-0249/ .. _round-robin DNS: https://en.wikipedia.org/wiki/Round-robin_DNS .. _sample application: https://github.com/crate/crate-sample-apps/tree/master/python -.. _setting up SSL: https://crate.io/docs/crate/reference/en/latest/admin/ssl.html -.. _socket timeout: https://docs.python.org/2/library/socket.html#socket.getdefaulttimeout -.. _SQLAlchemy: http://www.sqlalchemy.org/ -.. _tracebacks: https://docs.python.org/3/library/traceback.html -.. _urllib3 default socket options: https://urllib3.readthedocs.io/en/latest/reference/urllib3.connection.html#urllib3.connection.HTTPConnection +.. _sample application documentation: https://github.com/crate/crate-sample-apps/blob/master/python/documentation.md +.. _SQLAlchemy: https://www.sqlalchemy.org/ diff --git a/docs/getting-started.rst b/docs/getting-started.rst index af5e222a..699b1253 100644 --- a/docs/getting-started.rst +++ b/docs/getting-started.rst @@ -23,17 +23,19 @@ It may also work on earlier versions of Python. Install ======= +.. highlight:: sh + The CrateDB Python client is `available`_ as a `PyPI`_ package. To install the most recent driver version, including the SQLAlchemy dialect -extension, run: - -.. code-block:: sh +extension, run:: - sh$ pip install "crate[sqlalchemy]" --upgrade + pip install "crate[sqlalchemy]" --upgrade After that is done, you can import the library, like so: +.. code-block:: python + >>> from crate import client Interactive use @@ -43,11 +45,11 @@ Python provides a REPL_, also known as an interactive language shell. It's a handy way to experiment with code and try out new libraries. We recommend `iPython`_, which you can install, like so:: - sh$ pip install iPython + pip install iPython Once installed, you can start it up, like this:: - sh$ ipython + ipython From there, try importing the CrateDB Python client library and seeing how far you get with the built-in ``help()`` function (that can be called on any diff --git a/docs/index.rst b/docs/index.rst index da8df4c0..6c4dffc3 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -6,10 +6,10 @@ CrateDB Python Client A Python client library for `CrateDB`_. -This client library implements the `Python Database API 2.0`_ specification, -which defines a common interface for accessing databases in Python. +This client library implements the `Python Database API Specification v2.0`_ +(PEP 249), which defines a common interface for accessing databases in Python. -A :ref:`CrateDB dialect ` for `SQLAlchemy`_ is also included. +It also includes the :ref:`CrateDB dialect ` for `SQLAlchemy`_. .. NOTE:: @@ -19,7 +19,7 @@ A :ref:`CrateDB dialect ` for `SQLAlchemy`_ is also included. for a practical demonstration of this driver in use. For general help using the Python Database API or SQLAlchemy, please consult - `PEP 0249`_, the `SQLAlchemy tutorial`_, or the `SQLAlchemy documentation`_. + `PEP 249`_, the `SQLAlchemy tutorial`_, or the `SQLAlchemy documentation`_. .. SEEALSO:: @@ -42,9 +42,9 @@ A :ref:`CrateDB dialect ` for `SQLAlchemy`_ is also included. .. _CrateDB: https://crate.io/products/cratedb/ .. _documentation: https://github.com/crate/crate-sample-apps/blob/master/python/documentation.md .. _hosted on GitHub: https://github.com/crate/crate-python -.. _PEP 0249: http://www.python.org/dev/peps/pep-0249/ -.. _Python Database API 2.0: http://www.python.org/dev/peps/pep-0249/ +.. _PEP 249: https://www.python.org/dev/peps/pep-0249/ +.. _Python Database API Specification v2.0: https://www.python.org/dev/peps/pep-0249/ .. _sample application: https://github.com/crate/crate-sample-apps/tree/master/python -.. _SQLAlchemy tutorial: http://docs.sqlalchemy.org/en/latest/orm/tutorial.html -.. _SQLAlchemy: http://www.sqlalchemy.org/ -.. _SQLAlchemy documentation: http://docs.sqlalchemy.org/en/latest/ +.. _SQLAlchemy: https://www.sqlalchemy.org/ +.. _SQLAlchemy documentation: https://docs.sqlalchemy.org/ +.. _SQLAlchemy tutorial: https://docs.sqlalchemy.org/en/latest/orm/tutorial.html diff --git a/docs/query.rst b/docs/query.rst index 36f11983..a408f369 100644 --- a/docs/query.rst +++ b/docs/query.rst @@ -6,17 +6,16 @@ Query CrateDB .. NOTE:: - This page documents the CrateDB `Database API`_ client. + This page documents the CrateDB client, implementing the + `Python Database API Specification v2.0`_ (PEP 249). - For help using the `SQLAlchemy`_ dialect, consult - :ref:`the SQLAlchemy dialect documentation `. + For help using the `SQLAlchemy`_ dialect, consult + :ref:`the SQLAlchemy dialect documentation `. .. SEEALSO:: - Supplementary information about the CrateDB Database API client can be found - in the :ref:`data types appendix `. - - For general help using the Database API, consult `PEP 0249`_. + Supplementary information about the CrateDB Database API client can be found + in the :ref:`data types appendix `. .. rubric:: Table of contents @@ -31,7 +30,7 @@ Using a cursor After :ref:`connecting to CrateDB `, you can execute queries via a `database cursor`_. -Open a cursor like so:: +Open a cursor like so: >>> cursor = connection.cursor() @@ -46,38 +45,36 @@ Regular inserts Regular inserts are possible with the ``execute()`` method, like so: >>> cursor.execute( - ... """INSERT INTO locations (name, date, kind, position) - ... VALUES (?, ?, ?, ?)""", + ... "INSERT INTO locations (name, date, kind, position) VALUES (?, ?, ?, ?)", ... ("Einstein Cross", "2007-03-11", "Quasar", 7)) -Here, the values of the `tuple`_ (the second argument) are safely interpolated -into the query string (the first argument) where the ``?`` characters appear, -in the order they appear. +Here, the values of the :class:`py:tuple` (the second argument) are safely +interpolated into the query string (the first argument) where the ``?`` +characters appear, in the order they appear. .. WARNING:: - Never use string concatenation to build query strings. + Never use string concatenation to build query strings. - Always use the string interpolation feature of the client library (per the - above example) to guard against malicious input. + Always use the parameter interpolation feature of the client library to + guard against malicious input, as demonstrated in the example above. Bulk inserts ------------ -`Bulk inserts`_ are possible with the ``executemany()`` method, which takes a -`list`_ of tuples to insert:: +:ref:`Bulk inserts ` are possible with the +``executemany()`` method, which takes a :class:`py:list` of tuples to insert: >>> cursor.executemany( - ... """INSERT INTO locations (name, date, kind, position) - ... VALUES (?, ?, ?, ?)""", + ... "INSERT INTO locations (name, date, kind, position) VALUES (?, ?, ?, ?)", ... [('Cloverleaf', '2007-03-11', 'Quasar', 7), ... ('Old Faithful', '2007-03-11', 'Quasar', 7)]) [{'rowcount': 1}, {'rowcount': 1}] -The ``executemany()`` method returns a result `dictionary`_ for every tuple. -This dictionary always has a ``rowcount`` key, indicating how many rows were -inserted. If an error occures the ``rowcount`` value is ``-2`` and the -dictionary may additionally have an ``error_message`` key. +The ``executemany()`` method returns a result :class:`dictionary ` +for every tuple. This dictionary always has a ``rowcount`` key, indicating +how many rows were inserted. If an error occurs, the ``rowcount`` value is +``-2``, and the dictionary may additionally have an ``error_message`` key. .. _selects: @@ -87,7 +84,7 @@ Selecting data Executing a query ----------------- -Selects can be performed with the ``execute()`` method, like so:: +Selects can be performed with the ``execute()`` method, like so: >>> cursor.execute("SELECT name FROM locations WHERE name = ?", ("Algol",)) @@ -97,7 +94,7 @@ argument) where the ``?`` character appears. .. WARNING:: - As with :ref:`inserts `, always use string interpolation. + As with :ref:`inserts `, always use parameter interpolation. After executing a query, you can fetch the results using one of three fetch methods, detailed below. @@ -122,8 +119,8 @@ If no more rows are available, ``None`` is returned. .. TIP:: - The ``cursor`` object is an `iterator`_, and the ``fetchone()`` method is an - alias for ``next()``. + The ``cursor`` object is an :term:`py:iterator`, and the ``fetchone()`` + method is an alias for ``next()``. .. _fetchmany: @@ -150,7 +147,7 @@ list with one result row: .............. After executing a query, a ``fetchall()`` call on the cursor returns all -remaining rows:: +remaining rows: >>> cursor.execute("SELECT name FROM locations ORDER BY name") >>> cursor.fetchall() @@ -168,9 +165,9 @@ Result rows are lists, not dictionaries. Which means that they do use contain column names for keys. If you want to access column names, you must use ``cursor.description``. -The `DB API 2.0`_ specification `defines`_ seven description attributes per -column, but only the first one (column name) is supported by this library. All -other attributes are ``None``. +The `Python Database API Specification v2.0`_ `defines`_ seven description +attributes per column, but only the first one (column name) is supported by +this library. All other attributes are ``None``. Let's say you have a query like this: @@ -194,7 +191,7 @@ The cursor ``description`` might look like this: ('nullable_datetime', None, None, None, None, None, None), ('position', None, None, None, None, None, None)) -You can turn this into something more manageable with a `list comprehension`_:: +You can turn this into something more manageable with :ref:`py:tut-listcomps`: >>> [column[0] for column in cursor.description] ['date', 'datetime_tz', 'datetime_notz', ..., 'nullable_datetime', 'position'] @@ -207,8 +204,6 @@ The cursor object can optionally convert database types to native Python data types. There is a default implementation for the CrateDB data types ``IP`` and ``TIMESTAMP`` on behalf of the ``DefaultTypeConverter``. -:: - >>> from crate.client.converter import DefaultTypeConverter >>> from crate.client.cursor import Cursor >>> cursor = connection.cursor(converter=DefaultTypeConverter()) @@ -225,15 +220,14 @@ Custom data type conversion By providing a custom converter instance, you can define your own data type conversions. For investigating the list of available data types, please either inspect the ``DataType`` enum, or the documentation about the list of available -`CrateDB data type identifiers for the HTTP interface`_. +:ref:`CrateDB data type identifiers for the HTTP interface +`. This example creates and applies a simple custom converter for converging CrateDB's ``BOOLEAN`` type to Python's ``str`` type. It is using a simple converter function defined as ``lambda``, which assigns ``yes`` for boolean ``True``, and ``no`` otherwise. -:: - >>> from crate.client.converter import Converter, DataType >>> converter = Converter() @@ -256,8 +250,6 @@ desired time zone. For your reference, in the following examples, epoch 1658167836758 is ``Mon, 18 Jul 2022 18:10:36 GMT``. -:: - >>> import datetime >>> tz_mst = datetime.timezone(datetime.timedelta(hours=7), name="MST") >>> cursor = connection.cursor(time_zone=tz_mst) @@ -278,8 +270,6 @@ The available options are: Let's exercise all of them. -:: - >>> cursor.time_zone = datetime.timezone.utc >>> cursor.execute("SELECT datetime_tz FROM locations ORDER BY name") >>> cursor.fetchone() @@ -307,16 +297,7 @@ Let's exercise all of them. [datetime.datetime(2022, 7, 18, 23, 40, 36, 758000, tzinfo=datetime.timezone(datetime.timedelta(seconds=19800), '+0530'))] -.. _Bulk inserts: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html#bulk-operations -.. _CrateDB data type identifiers for the HTTP interface: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html#column-types -.. _Database API: http://www.python.org/dev/peps/pep-0249/ .. _database cursor: https://en.wikipedia.org/wiki/Cursor_(databases) -.. _DB API 2.0: http://www.python.org/dev/peps/pep-0249/ .. _defines: https://legacy.python.org/dev/peps/pep-0249/#description -.. _dictionary: https://docs.python.org/2/tutorial/datastructures.html#dictionaries -.. _iterator: https://wiki.python.org/moin/Iterator -.. _list comprehension: https://docs.python.org/2/tutorial/datastructures.html#list-comprehensions -.. _list: https://docs.python.org/2/library/stdtypes.html#sequence-types-str-unicode-list-tuple-bytearray-buffer-xrange -.. _PEP 0249: http://www.python.org/dev/peps/pep-0249/ -.. _SQLAlchemy: http://www.sqlalchemy.org/ -.. _tuple: https://docs.python.org/2/tutorial/datastructures.html#tuples-and-sequences +.. _Python Database API Specification v2.0: https://www.python.org/dev/peps/pep-0249/ +.. _SQLAlchemy: https://www.sqlalchemy.org/ diff --git a/docs/sqlalchemy.rst b/docs/sqlalchemy.rst index ff273f1a..fd19be30 100644 --- a/docs/sqlalchemy.rst +++ b/docs/sqlalchemy.rst @@ -18,17 +18,17 @@ Introduction `SQLAlchemy`_ is a popular `Object-Relational Mapping`_ (ORM) library for Python. -The CrateDB Python client library provides support for SQLAlchemy. A CrateDB -`dialect`_ is registered at installation time and can be used without further -configuration. +The CrateDB Python client library provides support for SQLAlchemy. An +:ref:`SQLAlchemy dialect ` for CrateDB is registered at +installation time and can be used without further configuration. -The CrateDB Python client library is validated to work with SQLAlchemy versions +The CrateDB SQLAlchemy dialect is validated to work with SQLAlchemy versions ``1.3`` and ``1.4``. .. SEEALSO:: - For general help using SQLAlchemy, consult the `SQLAlchemy tutorial`_ or the - `SQLAlchemy library`_. + For general help using SQLAlchemy, consult the :ref:`SQLAlchemy tutorial + ` or the `SQLAlchemy library`_. Supplementary information about the CrateDB SQLAlchemy dialect can be found in the :ref:`data types appendix `. @@ -47,8 +47,8 @@ Connecting Database URLs ------------- -An SQLAlchemy database is represented by special type of *Uniform Resource -Locator* (URL) called a `database URL`_. +In an SQLAlchemy context, database addresses are represented by *Uniform Resource +Locators* (URL_) called :ref:`sa:database_urls`. The simplest database URL for CrateDB looks like this:: @@ -62,7 +62,7 @@ A host string looks like this:: [:@]: Here, ```` is the hostname or IP address of the CrateDB node and -```` is a valid `psql.port`_ number. +```` is a valid :ref:`crate-reference:psql.port` number. When authentication is needed, the credentials can be optionally supplied using ``:@``. For connecting to an SSL-secured HTTP endpoint, you @@ -86,15 +86,14 @@ Getting a connection Create an engine ................ - You can connect to CrateDB using the ``create_engine`` method. This method -takes a `database URL`_. +takes a :ref:`database URL `. Import the ``sa`` module, like so: >>> import sqlalchemy as sa -To connect to ``localhost:4200``, you can do this:: +To connect to ``localhost:4200``, you can do this: >>> engine = sa.create_engine('crate://') @@ -105,7 +104,7 @@ To connect to ``crate-1.vm.example.com:4200``, you would do this: If your CrateDB cluster has multiple nodes, however, we recommend that you configure all of them. You can do that by specifying the ``crate://`` database URL and passing in a list of :ref:`host strings ` passed using -the ``connect_args`` argument, like so:: +the ``connect_args`` argument, like so: >>> engine = sa.create_engine('crate://', connect_args={ ... 'servers': ['198.51.100.1:4200', '198.51.100.2:4200'] @@ -114,10 +113,12 @@ the ``connect_args`` argument, like so:: When you do this, the Database API layer will use its :ref:`round-robin ` implementation. -The client validates `SSL server certificates`_ by default. For further -adjusting this behaviour, SSL verification options can be passed in by using -the ``connect_args`` dictionary. For example, use ``ca_cert`` for providing -a path to the CA certificate used for signing the server certificate:: +The client validates :ref:`SSL server certificates ` +by default. For further adjusting this behaviour, SSL verification options can +be passed in by using the ``connect_args`` dictionary. + +For example, use ``ca_cert`` for providing a path to the CA certificate used +for signing the server certificate: >>> engine = sa.create_engine( ... 'crate://', @@ -127,7 +128,7 @@ a path to the CA certificate used for signing the server certificate:: ... } ... ) -In order to disable SSL verification, use ``verify_ssl_cert = False``, like:: +In order to disable SSL verification, use ``verify_ssl_cert = False``, like: >>> engine = sa.create_engine( ... 'crate://', @@ -142,7 +143,7 @@ Get a session ............. Once you have an CrateDB ``engine`` set up, you can create and use an SQLAlchemy -``Session`` object to execute queries:: +``Session`` object to execute queries: >>> from sqlalchemy.orm import sessionmaker @@ -151,9 +152,7 @@ Once you have an CrateDB ``engine`` set up, you can create and use an SQLAlchemy .. SEEALSO:: - The SQLAlchemy has more documentation on `sessions`_. - -.. _sessions: http://docs.sqlalchemy.org/en/latest/orm/session_basics.html + SQLAlchemy has more documentation about this topic on :doc:`sa:orm/session_basics`. .. _tables: @@ -165,8 +164,8 @@ Tables Table definition ---------------- -Here is an example SQLAlchemy table definition using the `declarative -system`_: +Here is an example SQLAlchemy table definition using the :ref:`declarative +system `: >>> from sqlalchemy.ext import declarative >>> from crate.client.sqlalchemy import types @@ -199,7 +198,7 @@ system`_: In this example, we: -- Define a ``gen_key`` function that produces `UUIDs`_ +- Define a ``gen_key`` function that produces :py:mod:`UUIDs ` - Set up a ``Base`` class for the table - Create the ``Characters`` class for the ``characters`` table - Use the ``gen_key`` function to provide a default value for the ``id`` column @@ -220,8 +219,8 @@ In this example, we: .. SEEALSO:: - The SQLAlchemy documentation has more information about `working with - tables`_. + The SQLAlchemy documentation has more information about + :ref:`sa:metadata_describing`. Additional ``__table_args__`` @@ -240,8 +239,8 @@ table-wide attributes. The following attributes can optionally be configured: .. SEEALSO:: - The `CREATE TABLE`_ documentation contains more information on each of the - attributes. + The :ref:`CREATE TABLE ` documentation + contains more information on each of the attributes. ``_id`` as primary key @@ -261,7 +260,7 @@ A table schema like this "message" TEXT ) -would translate into the following declarative model:: +would translate into the following declarative model: >>> from sqlalchemy.schema import FetchedValue @@ -305,8 +304,8 @@ Objects are a common, and useful, data type when using CrateDB, so the CrateDB SQLAlchemy dialect provides a custom ``Object`` type extension for working with these values. -Here's how you might use the SQLAlchemy `Session`_ object to insert two -characters:: +Here's how you use the :doc:`SQLAlchemy Session ` to +insert two records: >>> # use the crate engine from earlier examples >>> Session = sessionmaker(bind=crate) @@ -330,15 +329,18 @@ characters:: .. NOTE:: The information we supply via the ``details`` column isn't defined in the - :ref:`original SQLAlchemy table definition `. These - details can be `specified`_ when you create the column in CrateDB, or you - can configure the column to support `dynamic values`_. + :ref:`original SQLAlchemy table definition ` schema. + These details can be specified as *object column policy* when you create + the column in CrateDB, you can either use the :ref:`STRICT column policy + `, or the :ref:`DYNAMIC column + policy `. .. NOTE:: Behind the scenes, if you update an ``Object`` property and ``commit`` that - change, the `UPDATE`_ statement sent to CrateDB will only include the data - necessary to update the changed subcolumns. + change, the :ref:`UPDATE ` statement sent + to CrateDB will only include the data necessary to update the changed + sub-columns. .. _objectarray: @@ -346,19 +348,20 @@ characters:: ............... In addition to the `Object`_ type, the CrateDB SQLAlchemy dialect also provides -a ``ObjectArray`` type, which is structured as a `list`_ of `dictionaries`_. +an ``ObjectArray`` type, which is structured as a :class:`py:list` of +:class:`dictionaries `. -Here's how you might set the value of an ``ObjectArray`` column:: +Here's how you might set the value of an ``ObjectArray`` column: >>> arthur.more_details = [{'foo': 1, 'bar': 10}, {'foo': 2}] >>> session.commit() -If you append an object, like this:: +If you append an object, like this: >>> arthur.more_details.append({'foo': 3}) >>> session.commit() -The resulting object will look like this:: +The resulting object will look like this: >>> arthur.more_details [{'foo': 1, 'bar': 10}, {'foo': 2}, {'foo': 3}] @@ -366,8 +369,8 @@ The resulting object will look like this:: .. CAUTION:: Behind the scenes, if you update an ``ObjectArray`` and ``commit`` that - change, the `UPDATE`_ statement sent to CrateDB will include all of the - ``ObjectArray`` data. + change, the :ref:`UPDATE ` statement + sent to CrateDB will include all of the ``ObjectArray`` data. .. _geopoint: .. _geoshape: @@ -380,7 +383,7 @@ The CrateDB SQLAlchemy dialect provides two geospatial types: - ``Geopoint``, which represents a longitude and latitude coordinate - ``Geoshape``, which is used to store geometric `GeoJSON geometry objects`_ -To use these types, you can create columns, like so:: +To use these types, you can create columns, like so: >>> class City(Base): ... @@ -390,16 +393,16 @@ To use these types, you can create columns, like so:: ... area = sa.Column(types.Geoshape) A geopoint can be created in multiple ways. Firstly, you can define it as a -tuple of ``(longitude, latitude)``:: +:py:class:`py:tuple` of ``(longitude, latitude)``: >>> point = (139.76, 35.68) -Secondly, you can define it as a geojson ``Point`` object:: +Secondly, you can define it as a geojson ``Point`` object: >>> from geojson import Point >>> point = Point(coordinates=(139.76, 35.68)) -To create a geoshape, you can use a geojson shape object, such as a ``Polygon``:: +To create a geoshape, you can use a geojson shape object, such as a ``Polygon``: >>> from geojson import Point, Polygon >>> area = Polygon( @@ -415,7 +418,7 @@ To create a geoshape, you can use a geojson shape object, such as a ``Polygon``: ... ] ... ) -You can then set the values of the ``Geopoint`` and ``Geoshape`` columns:: +You can then set the values of the ``Geopoint`` and ``Geoshape`` columns: >>> tokyo = City(name="Tokyo", coordinate=point, area=area) >>> session.add(tokyo) @@ -429,7 +432,8 @@ CrateDB. However, the newly inserted rows aren't immediately available for querying because the table index is only updated periodically (one second, by default, which is a short time for me and you, but a long time for your code). -You can request a `table refresh`_ to update the index manually:: +You can request a :ref:`table refresh ` to update +the index manually: >>> connection = engine.connect() >>> _ = connection.execute(text("REFRESH TABLE characters")) @@ -439,14 +443,14 @@ You can request a `table refresh`_ to update the index manually:: Newly inserted rows can still be queried immediately if a lookup by primary key is done. -Here's what a regular select might look like:: +Here's what a regular select might look like: >>> query = session.query(Character).order_by(Character.name) >>> [(c.name, c.details['gender']) for c in query] [('Arthur Dent', 'male'), ('Tricia McMillan', 'female')] You can also select a portion of each record, and this even works inside -`Object`_ columns:: +`Object`_ columns: >>> sorted(session.query(Character.details['gender']).all()) [('female',), ('male',)] @@ -457,7 +461,7 @@ You can also filter on attributes inside the `Object`_ column: >>> query.filter(Character.details['gender'] == 'male').all() [('Arthur Dent',)] -To filter on an `ObjectArray`_, you have to do something like this:: +To filter on an `ObjectArray`_, you have to do something like this: >>> from sqlalchemy.sql import operators @@ -465,8 +469,9 @@ To filter on an `ObjectArray`_, you have to do something like this:: >>> query.filter(Character.more_details['foo'].any(1, operator=operators.eq)).all() [(u'Arthur Dent',)] -Here, we're using the `any`_ method along with the `eq`_ Python `operator`_ to -match the value ``1`` against the ``foo`` key of any dictionary in the +Here, we're using SQLAlchemy's :py:meth:`any ` +method along with Python's :py:func:`py:operator.eq` function, in order to +match the value ``1`` against the key ``foo`` of any dictionary in the ``more_details`` list. Only one of the keys has to match for the row to be returned. @@ -478,7 +483,7 @@ key, like so: [1, 2, 3] Querying a key of an ``ObjectArray`` column will return all values for that key -for all matching rows:: +for all matching rows: >>> query = session.query(Character.more_details['foo']).order_by(Character.name) >>> query.all() @@ -525,12 +530,12 @@ Fulltext search Matching ........ -Fulltext Search in CrateDB is done with the `MATCH predicate`_. +Fulltext Search in CrateDB is done with the :ref:`crate-reference:predicates_match`. The CrateDB SQLAlchemy dialect provides a ``match`` function in the ``predicates`` module, which can be used to search one or multiple fields. -Here's an example use of the ``match`` function:: +Here's an example use of the ``match`` function: >>> from crate.client.sqlalchemy.predicates import match @@ -545,8 +550,8 @@ rows where the ``name_ft`` index matches the string ``Arthur``. .. NOTE:: To use fulltext searches on a column, an explicit fulltext index with an - analyzer must be created on the column. Consult the `fulltext indices - reference`_ for more information. + analyzer must be created on the column. Consult the documentation about + :ref:`crate-reference:fulltext-indices` for more information. The ``match`` function takes the following options:: @@ -571,8 +576,8 @@ The ``match`` function takes the following options:: .. SEEALSO:: - The ``MATCH`` predicate `arguments reference`_ has more in-depth - information. + The `arguments reference`_ of the :ref:`crate-reference:predicates_match` + has more in-depth information. :``term``: @@ -582,9 +587,11 @@ The ``match`` function takes the following options:: :``match_type``: *(optional)* - The `match type`_. + The :ref:`crate-reference:predicates_match_types`. - Determine how the ``term`` is applied and the `score`_ calculated. + Determine how the ``term`` is applied and the :ref:`_score + ` gets calculated. + See also `score usage`_. Here's an example:: @@ -614,7 +621,9 @@ The ``match`` function takes the following options:: Relevance ......... -To get the relevance of a matching row, the row `score`_ can be used. +To get the relevance of a matching row, the row :ref:`_score +` can be used. +See also `score usage`_. The score is relative to other result rows produced by your query. The higher the score, the more relevant the result row. @@ -628,7 +637,7 @@ The score is made available via the ``_score`` column, which is a virtual column, meaning that it doesn't exist on the source table, and in most cases, should not be included in your :ref:`table definition `. -You can select ``_score`` as part of a query, like this:: +You can select ``_score`` as part of a query, like this: >>> session.query(Character.name, '_score') \ ... .filter(match(Character.quote_ft, 'space')) \ @@ -642,36 +651,14 @@ the virtual column name as a string (``_score``) instead of using a defined column on the ``Character`` class. -.. _SQLAlchemy: http://www.sqlalchemy.org/ -.. _Object-Relational Mapping: https://en.wikipedia.org/wiki/Object-relational_mapping -.. _dialect: http://docs.sqlalchemy.org/en/latest/dialects/ -.. _SQLAlchemy tutorial: http://docs.sqlalchemy.org/en/latest/orm/tutorial.html -.. _database URL: http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls -.. _psql.port: https://crate.io/docs/crate/reference/en/latest/config/node.html#ports -.. _SSL server certificates: https://crate.io/docs/crate/reference/en/latest/admin/ssl.html -.. _SQLAlchemy library: http://www.sqlalchemy.org/library.html -.. _Database API: http://www.python.org/dev/peps/pep-0249/ -.. _declarative system: http://docs.sqlalchemy.org/en/latest/orm/extensions/declarative/ -.. _Session: http://docs.sqlalchemy.org/en/latest/orm/session.html -.. _specified: https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#strict -.. _dynamic values: https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#dynamic -.. _table refresh: https://crate.io/docs/crate/reference/en/latest/general/dql/refresh.html -.. _list: https://docs.python.org/3/library/stdtypes.html#lists -.. _dictionaries: https://docs.python.org/3/library/stdtypes.html#dict -.. _UPDATE: https://crate.io/docs/crate/reference/en/latest/general/dml.html#updating-data -.. _CREATE TABLE: https://crate.io/docs/crate/reference/en/latest/sql/statements/create-table.html -.. _eq: https://docs.python.org/2/library/operator.html#operator.eq -.. _operator: https://docs.python.org/2/library/operator.html -.. _any: http://docs.sqlalchemy.org/en/latest/core/type_basics.html#sqlalchemy.types.ARRAY.Comparator.any -.. _tuple: https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range -.. _count result rows: http://docs.sqlalchemy.org/en/14/orm/tutorial.html#counting -.. _MATCH predicate: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#match-predicate .. _arguments reference: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#arguments .. _boost values: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#arguments -.. _match type: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#predicates-match-types -.. _match options: https://crate.io/docs/stable/sql/fulltext.html#options -.. _fulltext indices reference: https://crate.io/docs/crate/reference/en/latest/general/ddl/fulltext-indices.html -.. _score: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#usage -.. _working with tables: http://docs.sqlalchemy.org/en/latest/core/metadata.html -.. _UUIDs: https://docs.python.org/3/library/uuid.html -.. _geojson geometry objects: https://tools.ietf.org/html/rfc7946#section-3.1 +.. _count result rows: https://docs.sqlalchemy.org/en/14/orm/tutorial.html#counting +.. _Database API: https://www.python.org/dev/peps/pep-0249/ +.. _geojson geometry objects: https://www.rfc-editor.org/rfc/rfc7946#section-3.1 +.. _match options: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#options +.. _Object-Relational Mapping: https://en.wikipedia.org/wiki/Object-relational_mapping +.. _score usage: https://crate.io/docs/crate/reference/en/latest/general/dql/fulltext.html#usage +.. _SQLAlchemy: https://www.sqlalchemy.org/ +.. _SQLAlchemy library: https://www.sqlalchemy.org/library.html +.. _URL: https://en.wikipedia.org/wiki/Uniform_Resource_Locator diff --git a/setup.py b/setup.py index 7070977a..0a127144 100644 --- a/setup.py +++ b/setup.py @@ -71,7 +71,7 @@ def read(path): 'stopit>=1.1.2,<2', 'flake8>=4,<5'], doc=['sphinx>=3.5,<6', - 'crate-docs-theme'], + 'crate-docs-theme>=0.26.5'], ), python_requires='>=3.4', package_data={'': ['*.txt']},