Skip to content

Commit

Permalink
Merge pull request #1376 from okfn/fix-filestore-docs
Browse files Browse the repository at this point in the history 
Fix filestore docs
  • Loading branch information
@kindly
kindly committed Dec 9, 2013
2 parents 6c5a5cb + 07449ae commit ca11b7a
Show file tree
Hide file tree
Showing 2 changed files with 142 additions and 56 deletions.
83 changes: 76 additions & 7 deletions doc/documentation-guidelines.rst
View file
Expand Up @@ -161,6 +161,12 @@ customize, initialize, color, etc. There's a list here:
https://wiki.ubuntu.com/EnglishTranslation/WordSubstitution


Spellcheck
==========

Please spellcheck documentation before merging it into master!


Commonly used terms
===================

Expand Down Expand Up @@ -342,13 +348,33 @@ As with Python code, try to limit all lines to a maximum of 79 characters.
Cross-references and links
==========================

Whenever mentioning another page or section in the docs, an external website, a
configuration setting, or a class, exception or function, etc. try to
cross-reference it. Using proper Sphinx cross-references is better than just
typing things like "see above/below" or "see section foo" because Sphinx
cross-refs are hyperlinked, and because if the thing you're referencing to gets
moved or deleted Sphinx will update the cross-reference or print a warning.


Cross-referencing to another file
---------------------------------

Use ``:doc:`` to cross-reference to other files by filename::

See :doc:`theming`
See :doc:`configuration`

If the file you're editing is in a subdir within the ``doc`` dir, you may need
to use an absolute reference (starting with a ``/``)::

see `Cross-referencing documents <http://sphinx-doc.org/markup/inline.html#cross-referencing-documents>`_
See :doc:`/configuration`

See `Cross-referencing documents <http://sphinx-doc.org/markup/inline.html#cross-referencing-documents>`_
for details.


Cross-referencing a section within a file
-----------------------------------------

Use ``:ref:`` to cross-reference to particular sections within the same or
another file. First you have to add a label before the section you want to
cross-reference to::
Expand All @@ -365,13 +391,59 @@ then from elsewhere cross-reference to the section like this::

see `Cross-referencing arbitrary locations <http://sphinx-doc.org/markup/inline.html#cross-referencing-arbitrary-locations>`_.

With both ``:doc:`` and ``:ref:`` if you want the link text to be different
from the title of the section you're referencing, do this::

Cross-referencing to CKAN config settings
-----------------------------------------

Whenever you mention a CKAN config setting, make it link to the docs for that
setting in :doc:`/configuration` by using ``:ref:`` and the name of the config
setting::

:ref:`ckan.site_title`

This works because all CKAN config settings are documented in
:doc:`/configuration`, and every setting has a Sphinx label that is exactly
the same as the name of the setting, for example::

.. _ckan.site_title:

ckan.site_title
^^^^^^^^^^^^^^^

Example::

ckan.site_title = Open Data Scotland

Default value: ``CKAN``

This sets the name of the site, as displayed in the CKAN web interface.

If you add a new config setting to CKAN, make sure to document like this it in
:doc:`configuration`.


Cross-referencing to a Python object
------------------------------------

Whenever you mention a Python function, method, object, class, exception, etc.
cross-reference it using a Sphinx domain object cross-reference.
See :ref:`Referencing other code objects`.


Changing the link text of a cross-reference
-------------------------------------------

With ``:doc:`` ``:ref:`` and other kinds of link, if you want the link text to
be different from the title of the thing you're referencing, do this::

:doc:`the theming document <theming>`

:ref:`the getting started section <getting-started>`


Cross-referencing to an external page
-------------------------------------

The syntax for linking to external URLs is slightly different from
cross-referencing, you have to add a trailing underscore::

Expand All @@ -386,9 +458,6 @@ or to define a URL once and then link to it in multiple places, do::

see `Hyperlinks <http://sphinx-doc.org/rest.html#hyperlinks>`_ for details.

Use ``:py:`` to reference other Python or JavaScript functions, modules,
classes, etc. See :ref:`Referencing other code objects`.


.. _sphinx substitutions:

Expand Down
115 changes: 66 additions & 49 deletions doc/filestore.rst
View file
@@ -1,19 +1,33 @@
==========================
FileStore and File Uploads
FileStore and file uploads
==========================

CKAN allows users to upload files directly to it against a resource or images
displayed against groups and organizations.
When enabled, CKAN's FileStore allows users to upload data files to CKAN
resources, and to upload logo images for groups and organizations. Users will
see an upload button when creating or updating a resource, group or
organization.

.. versionadded:: 2.2
Uploading logo images for groups and organizations was added in CKAN 2.2.

.. versionchanged:: 2.2
Previous versions of CKAN used to allow uploads to remote cloud hosting but
we have simplified this to only alow local file uploads (see below for
details on how to migrate). This is to give CKAN more control over the files
and make access control possible.
we have simplified this to only allow local file uploads (see
:ref:`filestore_21_to_22_migration` for details on how to migrate). This is
to give CKAN more control over the files and make access control possible.

.. seealso::

:ref:`datapusher`

Resource files linked-to from CKAN or uploaded to CKAN's FileStore can
also be pushed into CKAN's DataStore, which then enables data previews and
a data API for the resources.

-------------------------------------------
Setup File Uploads
-------------------------------------------

------------------
Setup file uploads
------------------

To setup CKAN's FileStore with local file storage:

Expand All @@ -23,16 +37,17 @@ To setup CKAN's FileStore with local file storage:
sudo mkdir -p |storage_path|
2. Add the following lines to your CKAN config file, after the ``[app:main]``
2. Add the following line to your CKAN config file, after the ``[app:main]``
line:

.. parsed-literal::
ckan.storage_dir = |storage_path|
ckan.storage_path = |storage_path|
3. Set the permissions of the ``storage_path``. For example if you're running
CKAN with Apache, then Apache's user (``www-data`` on Ubuntu) must have
read, write and execute permissions for the ``storage_path``:
3. Set the permissions of your :ref:`ckan.storage_path` directory.
For example if you're running CKAN with Apache, then Apache's user
(``www-data`` on Ubuntu) must have read, write and execute permissions for
the :ref:`ckan.storage_path`:

.. parsed-literal::
Expand All @@ -46,56 +61,58 @@ To setup CKAN's FileStore with local file storage:
|reload_apache|
-----------------------
FileStore Web Interface
-----------------------

Upload of files to storage is integrated directly into the Dataset creation
and editing system with files being associated to Resources.

-----------------------
-------------
FileStore API
-----------------------
-------------

.. versionchanged:: 2.2
The previous API has been deprecated although should still work if you where
using local file storage.

The api is part of the resource_create and resource_update action api
functions. You can post mutipart/form-data to the api and the key, value
pairs will treated as as if they are a json object.
The FileStore API was redesigned for CKAN 2.2.
The previous API has been deprecated.

Files can be uploaded to the FileStore using the
:py:func:`~ckan.logic.action.create.resource_create` and
:py:func:`~ckan.logic.action.update.resource_update` action API
functions. You can post multipart/form-data to the API and the key, value
pairs will treated as as if they are a JSON object.
The extra key ``upload`` is used to actually post the binary data.

Curl automatically puts the multipart-form-data heading when using the
``--form`` option:
For example, to create a new CKAN resource and upload a file to it using
`curl <http://curl.haxx.se/>`_:

.. parsed-literal::
.. parsed-literal::
curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_create' --form upload=@filetoupload --form package_id=my_dataset
curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_create' --form upload=@filetoupload --form package_id=my_dataset
The python requests library used the files parameter and automatically sets
the multipart/form-data header too:
(Curl automatically sends a ``multipart-form-data`` heading with you use the
``--form`` option.)

.. parsed-literal::
To create a new resource and upload a file to it using the Python library
`requests <http://python-requests.org/>`_:

import requests
requests.post('http://0.0.0.0:5000/api/action/resource_create',
data={"package_id":"my_dataset}",
headers={"X-CKAN-API-Key": "21a47217-6d7b-49c5-88f9-72ebd5a4d4bb"},
files=[('upload', file('/path/to/file/to/upload.csv'))])
.. parsed-literal::
With resource_update, if you want to override a file you just need
to set the upload field again::
import requests
requests.post('http://0.0.0.0:5000/api/action/resource_create',
data={"package_id":"my_dataset}",
headers={"X-CKAN-API-Key": "21a47217-6d7b-49c5-88f9-72ebd5a4d4bb"},
files=[('upload', file('/path/to/file/to/upload.csv'))])
(Requests automatically sends a ``multipart-form-data`` heading when you use the
``files=`` parameter.)

To overwrite an uploaded file with a new version of the file, post to the
:py:func:`~ckan.logic.action.update.resource_update` action and use the
``upload`` field::

curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_update' --form upload=@newfiletoupload --form id=resourceid

If you want to clear the upload and change it for a remote URL
there is special boolean field clear_upload to do this::
To replace an uploaded file with a link to a file at a remote URL, use the
``clear_upload`` field::

curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_update' --form url=http://expample.com --form clear_upload=true --form id=resourceid

It is also possible to have uploaded files (if of a suitable format) stored in
the DataStore which will then provides an API to the data. See :ref:`datapusher` for more details.

.. _filestore_21_to_22_migration:

--------------------------
Migration from 2.1 to 2.2
Expand All @@ -111,8 +128,8 @@ but if you want to move the remote storage documents to the local storage you wi
run the migration also.

In order to migrate make sure your CKAN instance is running as the script will
request the data from the instance using apis. You need to run the following
on the command line todo the migration::
request the data from the instance using APIs. You need to run the following
on the command line to do the migration::

paster db migrate-filestore

Expand Down

0 comments on commit ca11b7a

Please sign in to comment.