From ce3e42b9cb867d9f04163dd3834b0b7e113c4d09 Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Mon, 9 Dec 2013 13:19:13 +0100 Subject: [PATCH 1/6] Various FileStore docs fixes Correct storage_dir to storage_path. Correct case in titles (see doc guidelines). Fix Sphinx warnings - title underlines too long. When mentioning config settings, cross-reference them. When mentioning action functions, cross-reference them. api = API json = JSON python = Python Dedent parsed literals - the rst was formatted wrong so that it was outputting literals inside blockquotes. --- doc/filestore.rst | 51 ++++++++++++++++++++++++----------------------- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/doc/filestore.rst b/doc/filestore.rst index 7e38e31658c..cbd77c14168 100644 --- a/doc/filestore.rst +++ b/doc/filestore.rst @@ -1,5 +1,5 @@ ========================== -FileStore and File Uploads +FileStore and file uploads ========================== CKAN allows users to upload files directly to it against a resource or images @@ -11,9 +11,9 @@ displayed against groups and organizations. details on how to migrate). This is to give CKAN more control over the files and make access control possible. -------------------------------------------- -Setup File Uploads -------------------------------------------- +------------------ +Setup file uploads +------------------ To setup CKAN's FileStore with local file storage: @@ -28,11 +28,11 @@ To setup CKAN's FileStore with local file storage: .. parsed-literal:: - ckan.storage_dir = |storage_path| + ckan.storage_path = |storage_path| -3. Set the permissions of the ``storage_path``. For example if you're running +3. Set the permissions of the :ref:`ckan.storage_path`. For example if you're running CKAN with Apache, then Apache's user (``www-data`` on Ubuntu) must have - read, write and execute permissions for the ``storage_path``: + read, write and execute permissions for the :ref:`ckan.storage_path`: .. parsed-literal:: @@ -47,45 +47,46 @@ To setup CKAN's FileStore with local file storage: ----------------------- -FileStore Web Interface +FileStore web interface ----------------------- Upload of files to storage is integrated directly into the Dataset creation and editing system with files being associated to Resources. ------------------------ +------------- FileStore API ------------------------ +------------- .. versionchanged:: 2.2 The previous API has been deprecated although should still work if you where using local file storage. -The api is part of the resource_create and resource_update action api -functions. You can post mutipart/form-data to the api and the key, value -pairs will treated as as if they are a json object. +The API is part of the :py:func:`~ckan.logic.action.create.resource_create` and +:py:func:`~ckan.logic.action.update.resource_update` action API +functions. You can post mutipart/form-data to the API and the key, value +pairs will treated as as if they are a JSON object. The extra key ``upload`` is used to actually post the binary data. Curl automatically puts the multipart-form-data heading when using the ``--form`` option: - .. parsed-literal:: +.. parsed-literal:: - curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_create' --form upload=@filetoupload --form package_id=my_dataset + curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_create' --form upload=@filetoupload --form package_id=my_dataset -The python requests library used the files parameter and automatically sets +The Python requests library used the files parameter and automatically sets the multipart/form-data header too: - .. parsed-literal:: +.. parsed-literal:: - import requests - requests.post('http://0.0.0.0:5000/api/action/resource_create', - data={"package_id":"my_dataset}", - headers={"X-CKAN-API-Key": "21a47217-6d7b-49c5-88f9-72ebd5a4d4bb"}, - files=[('upload', file('/path/to/file/to/upload.csv'))]) + import requests + requests.post('http://0.0.0.0:5000/api/action/resource_create', + data={"package_id":"my_dataset}", + headers={"X-CKAN-API-Key": "21a47217-6d7b-49c5-88f9-72ebd5a4d4bb"}, + files=[('upload', file('/path/to/file/to/upload.csv'))]) -With resource_update, if you want to override a file you just need -to set the upload field again:: +With :py:func:`~ckan.logic.action.update.resource_update`, if you want to +override a file you just need to set the upload field again:: curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_update' --form upload=@newfiletoupload --form id=resourceid @@ -111,7 +112,7 @@ but if you want to move the remote storage documents to the local storage you wi run the migration also. In order to migrate make sure your CKAN instance is running as the script will -request the data from the instance using apis. You need to run the following +request the data from the instance using APIs. You need to run the following on the command line todo the migration:: paster db migrate-filestore From c4a71abf60b31594c2fef7488d36b71c4e139461 Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Mon, 9 Dec 2013 13:27:29 +0100 Subject: [PATCH 2/6] Fix a cross-reference If you're going to mention another section in the docs, cross-reference it properly. This way if that section gets moved or deleted, a Sphinx error will remind you to update the cross-reference (or Sphinx will do it automatically). Also users get a nice hyperlink. --- doc/filestore.rst | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/doc/filestore.rst b/doc/filestore.rst index cbd77c14168..60f7a066da2 100644 --- a/doc/filestore.rst +++ b/doc/filestore.rst @@ -7,9 +7,9 @@ displayed against groups and organizations. .. versionchanged:: 2.2 Previous versions of CKAN used to allow uploads to remote cloud hosting but - we have simplified this to only alow local file uploads (see below for - details on how to migrate). This is to give CKAN more control over the files - and make access control possible. + we have simplified this to only alow local file uploads (see + :ref:`filestore_21_to_22_migration` for details on how to migrate). This is + to give CKAN more control over the files and make access control possible. ------------------ Setup file uploads @@ -98,6 +98,8 @@ there is special boolean field clear_upload to do this:: It is also possible to have uploaded files (if of a suitable format) stored in the DataStore which will then provides an API to the data. See :ref:`datapusher` for more details. +.. _filestore_21_to_22_migration: + -------------------------- Migration from 2.1 to 2.2 -------------------------- From eb7ec826da38b0f97e35bc5a736f4b331b2c4216 Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Mon, 9 Dec 2013 13:50:18 +0100 Subject: [PATCH 3/6] General edit of new FileStore docs Edit text for clarity, simplicity. --- doc/filestore.rst | 55 ++++++++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 24 deletions(-) diff --git a/doc/filestore.rst b/doc/filestore.rst index 60f7a066da2..0c9e3eb4f1c 100644 --- a/doc/filestore.rst +++ b/doc/filestore.rst @@ -2,8 +2,13 @@ FileStore and file uploads ========================== -CKAN allows users to upload files directly to it against a resource or images -displayed against groups and organizations. +When enabled, CKAN's FileStore allows users to upload data files to CKAN +resources, and to upload logo images for groups and organizations. Users will +see an upload button when creating or updating a resource, group or +organization. + +.. versionadded:: 2.2 + Uploading logo images for groups and organizations was added in CKAN 2.2. .. versionchanged:: 2.2 Previous versions of CKAN used to allow uploads to remote cloud hosting but @@ -23,16 +28,17 @@ To setup CKAN's FileStore with local file storage: sudo mkdir -p |storage_path| -2. Add the following lines to your CKAN config file, after the ``[app:main]`` +2. Add the following line to your CKAN config file, after the ``[app:main]`` line: .. parsed-literal:: ckan.storage_path = |storage_path| -3. Set the permissions of the :ref:`ckan.storage_path`. For example if you're running - CKAN with Apache, then Apache's user (``www-data`` on Ubuntu) must have - read, write and execute permissions for the :ref:`ckan.storage_path`: +3. Set the permissions of your :ref:`ckan.storage_path` directory. + For example if you're running CKAN with Apache, then Apache's user + (``www-data`` on Ubuntu) must have read, write and execute permissions for + the :ref:`ckan.storage_path`: .. parsed-literal:: @@ -46,36 +52,33 @@ To setup CKAN's FileStore with local file storage: |reload_apache| ------------------------ -FileStore web interface ------------------------ - -Upload of files to storage is integrated directly into the Dataset creation -and editing system with files being associated to Resources. - ------------- FileStore API ------------- .. versionchanged:: 2.2 - The previous API has been deprecated although should still work if you where - using local file storage. + The FileStore API was redesigned for CKAN 2.2. + The previous API has been deprecated. -The API is part of the :py:func:`~ckan.logic.action.create.resource_create` and +Files can be uploaded to the FileStore using the +:py:func:`~ckan.logic.action.create.resource_create` and :py:func:`~ckan.logic.action.update.resource_update` action API functions. You can post mutipart/form-data to the API and the key, value pairs will treated as as if they are a JSON object. The extra key ``upload`` is used to actually post the binary data. -Curl automatically puts the multipart-form-data heading when using the -``--form`` option: +For example, to create a new CKAN resource and upload a file to it using +`curl `_: .. parsed-literal:: curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_create' --form upload=@filetoupload --form package_id=my_dataset -The Python requests library used the files parameter and automatically sets -the multipart/form-data header too: +(Curl automatically sends a ``multipart-form-data`` heading with you use the +``--form`` option.) + +To create a new resource and upload a file to it using the Python library +`requests `_: .. parsed-literal:: @@ -85,13 +88,17 @@ the multipart/form-data header too: headers={"X-CKAN-API-Key": "21a47217-6d7b-49c5-88f9-72ebd5a4d4bb"}, files=[('upload', file('/path/to/file/to/upload.csv'))]) -With :py:func:`~ckan.logic.action.update.resource_update`, if you want to -override a file you just need to set the upload field again:: +(Requests automatically sends a ``multipart-form-data`` heading when you use the +``files=`` parameter.) + +To overwrite an uploaded file with a new version of the file, post to the +:py:func:`~ckan.logic.action.update.resource_update` action and use the +``upload`` field:: curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_update' --form upload=@newfiletoupload --form id=resourceid -If you want to clear the upload and change it for a remote URL -there is special boolean field clear_upload to do this:: +To replace an uploaded file with a link to a file at a remote URL, use the +``clear_upload`` field:: curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_update' --form url=http://expample.com --form clear_upload=true --form id=resourceid From 0ddce2631877527df6824ec1778bd66e85770de1 Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Mon, 9 Dec 2013 13:55:07 +0100 Subject: [PATCH 4/6] Change DataStore ref in FileStore docs to seealso It'll be good if we can use this style (a Sphinx .. seealso::, near the top of the page) whenever we want to say "also see this page" in the docs. That'll make it a lot more predictable and easier for users to find, when one feature is connected to another. --- doc/filestore.rst | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/doc/filestore.rst b/doc/filestore.rst index 0c9e3eb4f1c..156e085ed8d 100644 --- a/doc/filestore.rst +++ b/doc/filestore.rst @@ -16,6 +16,15 @@ organization. :ref:`filestore_21_to_22_migration` for details on how to migrate). This is to give CKAN more control over the files and make access control possible. +.. seealso:: + + :ref:`datapusher` + + Resource files linked-to from CKAN or uploaded to CKAN's FileStore can + also be pushed into CKAN's DataStore, which then enables data previews and + a data API for the resources. + + ------------------ Setup file uploads ------------------ @@ -102,8 +111,6 @@ To replace an uploaded file with a link to a file at a remote URL, use the curl -H'Authorization: your-api-key' 'http://yourhost/api/action/resource_update' --form url=http://expample.com --form clear_upload=true --form id=resourceid -It is also possible to have uploaded files (if of a suitable format) stored in -the DataStore which will then provides an API to the data. See :ref:`datapusher` for more details. .. _filestore_21_to_22_migration: From 2c53e5b7a197345f952708c3f05a369b30f583bc Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Mon, 9 Dec 2013 14:03:02 +0100 Subject: [PATCH 5/6] Spellcheck new FileStore docs --- doc/filestore.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/filestore.rst b/doc/filestore.rst index 156e085ed8d..f57057ebd97 100644 --- a/doc/filestore.rst +++ b/doc/filestore.rst @@ -12,7 +12,7 @@ organization. .. versionchanged:: 2.2 Previous versions of CKAN used to allow uploads to remote cloud hosting but - we have simplified this to only alow local file uploads (see + we have simplified this to only allow local file uploads (see :ref:`filestore_21_to_22_migration` for details on how to migrate). This is to give CKAN more control over the files and make access control possible. @@ -72,7 +72,7 @@ FileStore API Files can be uploaded to the FileStore using the :py:func:`~ckan.logic.action.create.resource_create` and :py:func:`~ckan.logic.action.update.resource_update` action API -functions. You can post mutipart/form-data to the API and the key, value +functions. You can post multipart/form-data to the API and the key, value pairs will treated as as if they are a JSON object. The extra key ``upload`` is used to actually post the binary data. @@ -129,7 +129,7 @@ run the migration also. In order to migrate make sure your CKAN instance is running as the script will request the data from the instance using APIs. You need to run the following -on the command line todo the migration:: +on the command line to do the migration:: paster db migrate-filestore From 07449ae59acf74bf84433eeab117bf2d26c96d8a Mon Sep 17 00:00:00 2001 From: Sean Hammond Date: Mon, 9 Dec 2013 14:48:28 +0100 Subject: [PATCH 6/6] Add some details to the documentation guidelines --- doc/documentation-guidelines.rst | 83 +++++++++++++++++++++++++++++--- 1 file changed, 76 insertions(+), 7 deletions(-) diff --git a/doc/documentation-guidelines.rst b/doc/documentation-guidelines.rst index 6014c645016..ddfcc23b19f 100644 --- a/doc/documentation-guidelines.rst +++ b/doc/documentation-guidelines.rst @@ -161,6 +161,12 @@ customize, initialize, color, etc. There's a list here: https://wiki.ubuntu.com/EnglishTranslation/WordSubstitution +Spellcheck +========== + +Please spellcheck documentation before merging it into master! + + Commonly used terms =================== @@ -342,13 +348,33 @@ As with Python code, try to limit all lines to a maximum of 79 characters. Cross-references and links ========================== +Whenever mentioning another page or section in the docs, an external website, a +configuration setting, or a class, exception or function, etc. try to +cross-reference it. Using proper Sphinx cross-references is better than just +typing things like "see above/below" or "see section foo" because Sphinx +cross-refs are hyperlinked, and because if the thing you're referencing to gets +moved or deleted Sphinx will update the cross-reference or print a warning. + + +Cross-referencing to another file +--------------------------------- + Use ``:doc:`` to cross-reference to other files by filename:: - See :doc:`theming` + See :doc:`configuration` + +If the file you're editing is in a subdir within the ``doc`` dir, you may need +to use an absolute reference (starting with a ``/``):: -see `Cross-referencing documents `_ + See :doc:`/configuration` + +See `Cross-referencing documents `_ for details. + +Cross-referencing a section within a file +----------------------------------------- + Use ``:ref:`` to cross-reference to particular sections within the same or another file. First you have to add a label before the section you want to cross-reference to:: @@ -365,13 +391,59 @@ then from elsewhere cross-reference to the section like this:: see `Cross-referencing arbitrary locations `_. -With both ``:doc:`` and ``:ref:`` if you want the link text to be different -from the title of the section you're referencing, do this:: + +Cross-referencing to CKAN config settings +----------------------------------------- + +Whenever you mention a CKAN config setting, make it link to the docs for that +setting in :doc:`/configuration` by using ``:ref:`` and the name of the config +setting:: + + :ref:`ckan.site_title` + +This works because all CKAN config settings are documented in +:doc:`/configuration`, and every setting has a Sphinx label that is exactly +the same as the name of the setting, for example:: + + .. _ckan.site_title: + + ckan.site_title + ^^^^^^^^^^^^^^^ + + Example:: + + ckan.site_title = Open Data Scotland + + Default value: ``CKAN`` + + This sets the name of the site, as displayed in the CKAN web interface. + +If you add a new config setting to CKAN, make sure to document like this it in +:doc:`configuration`. + + +Cross-referencing to a Python object +------------------------------------ + +Whenever you mention a Python function, method, object, class, exception, etc. +cross-reference it using a Sphinx domain object cross-reference. +See :ref:`Referencing other code objects`. + + +Changing the link text of a cross-reference +------------------------------------------- + +With ``:doc:`` ``:ref:`` and other kinds of link, if you want the link text to +be different from the title of the thing you're referencing, do this:: :doc:`the theming document ` :ref:`the getting started section ` + +Cross-referencing to an external page +------------------------------------- + The syntax for linking to external URLs is slightly different from cross-referencing, you have to add a trailing underscore:: @@ -386,9 +458,6 @@ or to define a URL once and then link to it in multiple places, do:: see `Hyperlinks `_ for details. -Use ``:py:`` to reference other Python or JavaScript functions, modules, -classes, etc. See :ref:`Referencing other code objects`. - .. _sphinx substitutions: