Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions docs/_extra/robots.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,2 @@
User-agent: *
Disallow: /

Sitemap: https://crate.io/docs/python/en/latest/site.xml
User-agent: *
3 changes: 0 additions & 3 deletions docs/appendices/data-types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,3 @@ __ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#o
__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#array
__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-point
__ https://crate.io/docs/crate/reference/en/latest/general/ddl/data-types.html#geo-shape

.. _json: https://docs.python.org/3/library/json.html
.. _HTTP endpoint: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html
47 changes: 21 additions & 26 deletions docs/blobs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@
Blobs
=====

The CrateDB Python client library provides full access to the powerful `blob
storage capabilities`_ of your CrateDB cluster.
The CrateDB Python client library provides full access to the powerful
:ref:`blob storage capabilities <crate-reference:blob_support>` of your
CrateDB cluster.

.. rubric:: Table of contents

Expand All @@ -25,12 +26,14 @@ For the sake of this example, we will do the following:
>>> connection = client.connect("http://localhost:4200/")

This is a simple connection that connects to a CrateDB node running on
the local host with the `HTTP endpoint`_ listening on port 4200 (the default).
the local host with the :ref:`crate-reference:interface-http` listening
on port 4200 (the default).

To work with blobs in CrateDB, you must specifically `create blob tables`_.
To work with blobs in CrateDB, you must specifically create
:ref:`blob tables <crate-reference:blob_support>`.

The CrateDB Python client allows you to interact with these blob tables via a
blob container, which you can create like this::
blob container, which you can create like this:

>>> blob_container = connection.get_blob_container('my_blobs')
>>> blob_container
Expand All @@ -53,31 +56,31 @@ produce bytes when read.
What is a file-like object? Well, to put it simply, any object that provides a
``read()`` method.

The stream objects provided by the Python standard library `io`_ module and
`tempfile`_ module are the most commonly used file-like objects.
The stream objects provided by the Python standard library :mod:`py:io` and
:mod:`py:tempfile` modules are the most commonly used file-like objects.

The `StringIO`_ class is not suitable, as it produces Unicode strings when
read. But you can easily encode a Unicode string and feed it to a `BytesIO`_
The :class:`py:io.StringIO` class is not suitable, as it produces Unicode strings when
read. But you can easily encode a Unicode string and feed it to a :class:`py:io.BytesIO`
object.

Here's a trivial example::
Here's a trivial example:

>>> import io
>>> bytestream = "An example sentence.".encode("utf8")
>>> file = io.BytesIO(bytestream)

This file can then be uploaded to the blob table using the ``put`` method::
This file can then be uploaded to the blob table using the ``put`` method:

>>> blob_container.put(file)
'6f10281ad07d4a35c6ec2f993e6376032b77181d'

Notice that this method computes and returns a `SHA-1 digest`_. This is
Notice that this method computes and returns an `SHA-1 digest`_. This is
necessary for attempting to save the blob to CrateDB.

If you already have the SHA-1 digest computed, or are able to compute it as part
of an existing read, this may improve the performance of your application.

If you pass in a SHA-1 digest, it will not be recomputed::
If you pass in a SHA-1 digest, it will not be recomputed:

>>> file.seek(0) # seek to the beginning before attempting to re-upload

Expand Down Expand Up @@ -123,12 +126,12 @@ You can get the blob, with the ``get`` method, like so:
>>> blob_generator = blob_container.get(digest)

Blobs are read in chunks. The default size of these chunks is 128 kilobytes,
but this can be changed by suppling the desired chunk size to the ``get``
method, like so::
but this can be changed by supplying the desired chunk size to the ``get``
method, like so:

>>> res = blob_container.get(digest, 1024 * 128)

The ``blob`` object is a Python `generator`_, meaning that you can call
The ``blob`` object is a Python :term:`py:generator`, meaning that you can call
``next(blob)`` for each new chunk you want to read, until you encounter a
``StopIteration`` exception.

Expand All @@ -143,7 +146,7 @@ generator is like so:
Delete blobs
------------

You can delete a blob with the ``delete`` method and the blob digest, like so::
You can delete a blob with the ``delete`` method and the blob digest, like so:

>>> blob_container.delete(digest)
True
Expand All @@ -156,12 +159,4 @@ We can verify that, like so:
>>> blob_container.exists(digest)
False

.. _blob storage capabilities: https://crate.io/docs/crate/reference/en/latest/general/blobs.html
.. _BytesIO: https://docs.python.org/3/library/io.html#binary-i-o
.. _create blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html#creating-a-table-for-blobs
.. _generator: https://stackoverflow.com/a/1756156
.. _HTTP endpoint: https://crate.io/docs/crate/reference/en/latest/interfaces/http.html
.. _io: https://docs.python.org/3/library/io.html
.. _SHA-1 digest: https://docs.python.org/3/library/hashlib.html
.. _StringIO: https://docs.python.org/3/library/io.html#io.StringIO
.. _tempfile: https://docs.python.org/3/library/tempfile.html
.. _SHA-1 digest: https://en.wikipedia.org/wiki/SHA-1
25 changes: 11 additions & 14 deletions docs/by-example/blob.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,19 @@
Blob container API
==================

The connection object provides a convenience API for easy access to `blob
tables`_.
The connection object provides a convenience API for easy access to
:ref:`blob tables <crate-reference:blob_support>`.


Get blob container handle
=========================

Create a connection::
Create a connection:

>>> from crate.client import connect
>>> client = connect([crate_host])

Get a blob container::
Get a blob container:

>>> container = client.get_blob_container('myfiles')

Expand All @@ -26,15 +26,15 @@ The container allows to store a blob without explicitly providing the hash
for the blob. This feature is possible if the blob is provided as a seekable
stream like object.

Store a ``StringIO`` stream::
Store a ``StringIO`` stream:

>>> from io import BytesIO
>>> f = BytesIO(b'StringIO data')
>>> stringio_bob = container.put(f)
>>> stringio_bob
'0cd4511d696823779692484029f234471cd21f28'

Store from a file::
Store from a file:

>>> from tempfile import TemporaryFile
>>> f = TemporaryFile()
Expand All @@ -46,7 +46,7 @@ Store from a file::
>>> f.close()

If the blob data is not provided as a seekable stream the hash must be
provided explicitly::
provided explicitly:

>>> import hashlib
>>> string_data = b'String data'
Expand All @@ -67,7 +67,7 @@ Check for existence
Retrieve blobs
==============

Blobs can be retrieved using its hash::
Blobs can be retrieved using its hash:

>>> blob_stream = container.get(string_blob)
>>> blob_stream
Expand All @@ -80,24 +80,21 @@ Blobs can be retrieved using its hash::
Delete blobs
============

Blobs can be deleted using its hash::
Blobs can be deleted using its hash:

>>> container.delete(string_blob)
True
>>> container.exists(string_blob)
False

Trying to delete a not existing blob::
Trying to delete a not existing blob:

>>> container.delete(string_blob)
False

Close connection
================

Close the connection to clear the connection pool::
Close the connection to clear the connection pool:

>>> client.close()


.. _blob tables: https://crate.io/docs/crate/reference/en/latest/general/blobs.html
Loading