diff --git a/ckanext/datastore/templates/datastore/snippets/dictionary_form.html b/ckanext/datastore/templates/datastore/snippets/dictionary_form.html
index fad1d662df7..41266c9aae9 100644
--- a/ckanext/datastore/templates/datastore/snippets/dictionary_form.html
+++ b/ckanext/datastore/templates/datastore/snippets/dictionary_form.html
@@ -1,16 +1,25 @@
{% import 'macros/form.html' as form %}
{{ _( "Field {num}.").format(num=position) }} {{ field.id }} ({{ field.type }})
- {{ form.select('info__' ~ position ~ '__type_override',
- label=_('Type Override'), options=[
- {'name': '', 'value': ''},
- {'name': 'text', 'value': 'text'},
- {'name': 'numeric', 'value': 'numeric'},
- {'name': 'timestamp', 'value': 'timestamp'},
- ], selected=field.get('info', {}).get('type_override', '')) }}
- {{ form.input('info__' ~ position ~ '__label',
- label=_('Label'), id='field-f' ~ position ~ 'label',
- value=field.get('info', {}).get('label', ''), classes=['control-full']) }}
- {{ form.markdown('info__' ~ position ~ '__notes',
- label=_('Description'), id='field-d' ~ position ~ 'notes',
- value=field.get('info', {}).get('notes', '')) }}
+
+{#
+ Data Dictionary fields may be added this snippet. New fields following
+ the 'info__' ~ position ~ '__namegoeshere' convention will be saved
+ as part of the "info" object on the column.
+#}
+
+{{ form.select('info__' ~ position ~ '__type_override',
+ label=_('Type Override'), options=[
+ {'name': '', 'value': ''},
+ {'name': 'text', 'value': 'text'},
+ {'name': 'numeric', 'value': 'numeric'},
+ {'name': 'timestamp', 'value': 'timestamp'},
+ ], selected=field.get('info', {}).get('type_override', '')) }}
+
+{{ form.input('info__' ~ position ~ '__label',
+ label=_('Label'), id='field-f' ~ position ~ 'label',
+ value=field.get('info', {}).get('label', ''), classes=['control-full']) }}
+
+{{ form.markdown('info__' ~ position ~ '__notes',
+ label=_('Description'), id='field-d' ~ position ~ 'notes',
+ value=field.get('info', {}).get('notes', '')) }}
diff --git a/doc/maintaining/datastore.rst b/doc/maintaining/datastore.rst
index c4da16b5d16..9d058b49c60 100644
--- a/doc/maintaining/datastore.rst
+++ b/doc/maintaining/datastore.rst
@@ -233,6 +233,49 @@ alongside CKAN.
To install this please look at the docs here: http://docs.ckan.org/projects/datapusher
+.. _data_dictionary:
+
+---------------
+Data Dictionary
+---------------
+
+DataStore columns may be described with a Data Dictionary. A Data Dictionary tab
+will appear when editing any resource with a DataStore table.
+The Data Dictionary form allows entering the following values for
+each column:
+
+* **Type Override:** the type to be used the next time DataPusher is run to load
+ data into this column
+* **Label:** a human-friendly label for this column
+* **Description:** a full description for this column in markdown format
+
+Extension developers may add new fields to this form by overriding the default
+Data Dictionary form template ``datastore/snippets/dictionary_form.html``.
+
+The Data Dictionary is set through the API as part of the :ref:`fields` passed
+to :meth:`~ckanext.datastore.logic.action.datastore_create` and
+returned from :meth:`~ckanext.datastore.logic.action.datastore_search`.
+
+
+.. _dump:
+
+---------------------
+Downloading Resources
+---------------------
+
+A DataStore resource can be downloaded in the `CSV`_ file format from ``{CKAN-URL}/datastore/dump/{RESOURCE-ID}``.
+
+For an Excel-compatible CSV file use ``{CKAN-URL}/datastore/dump/{RESOURCE-ID}?bom=true``.
+
+Other formats supported include tab-separated values (``?format=tsv``),
+JSON (``?format=json``) and XML (``?format=xml``). E.g. to download an Excel-compatible
+tab-separated file use
+``{CKAN-URL}/datastore/dump/{RESOURCE-ID}?format=tsv&bom=true``.
+
+.. _CSV: https://en.wikipedia.org/wiki/Comma-separated_values
+
+
+
-----------------
The DataStore API
-----------------
@@ -279,23 +322,6 @@ API reference
:members:
-.. _dump:
-
-Download resource
------------------
-
-A DataStore resource can be downloaded in the `CSV`_ file format from ``{CKAN-URL}/datastore/dump/{RESOURCE-ID}``.
-
-For an Excel-compatible CSV file use ``{CKAN-URL}/datastore/dump/{RESOURCE-ID}?bom=true``.
-
-Other formats supported include tab-separated values (``?format=tsv``),
-JSON (``?format=json``) and XML (``?format=xml``). E.g. to download an Excel-compatible
-tab-separated file use
-``{CKAN-URL}/datastore/dump/{RESOURCE-ID}?format=tsv&bom=true``.
-
-.. _CSV: https://en.wikipedia.org/wiki/Comma-separated_values
-
-
.. _fields:
Fields
@@ -304,22 +330,40 @@ Fields
Fields define the column names and the type of the data in a column. A field is defined as follows::
{
- "id": # a string which defines the column name
+ "id": # the column name (required)
"type": # the data type for the column
+ "info": {
+ "label": # human-readable label for column
+ "notes": # markdown description of column
+ "type_override": # type for datapusher to use when importing data
+ ...: # other user-defined fields
+ }
}
-Field **types are optional** and will be guessed by the DataStore from the provided data. However, setting the types ensures that future inserts will not fail because of wrong types. See :ref:`valid-types` for details on which types are valid.
+Field types not provided will be guessed based on the first row of provided data.
+Set the types to ensure that future inserts will not fail because of an incorrectly
+guessed type. See :ref:`valid-types` for details on which types are valid.
+
+Extra ``"info"`` field values will be stored along with the column. ``"label"``,
+``"notes"`` and ``"type_override"`` can be managed from the default :ref:`data_dictionary`
+form. Additional fields can be stored by customizing the Data Dictionary form or by
+passing their values to the API directly.
Example::
[
{
- "id": "foo",
- "type": "int4"
+ "id": "code_number",
+ "type": "numeric"
},
{
- "id": "bar"
- # type is optional
+ "id": "description"
+ "type": "text",
+ "info": {
+ "label": "Description",
+ "notes": "A brief usage description for this code",
+ "example": "Used for temporary service interruptions"
+ }
}
]
@@ -331,19 +375,21 @@ Records
A record is the data to be inserted in a DataStore resource and is defined as follows::
{
- "": # data to be set
- # .. more data
+ column_1_id: value_1,
+ columd_2_id: value_2,
+ ...
}
Example::
[
{
- "foo": 100,
- "bar": "Here's some text"
+ "code_number": 10,
+ "description": "Submitted successfully"
},
{
- "foo": 42
+ "code_number": 42,
+ "description": "In progress"
}
]
@@ -437,8 +483,10 @@ name
oid
The PostgreSQL object ID of the table that belongs to name.
+
+-------------------
Extending DataStore
-===================
+-------------------
Starting from CKAN version 2.7, backend used in DataStore can be replaced with custom one. For this purpose, custom extension must implement `ckanext.datastore.interfaces.IDatastoreBackend`, which provides one method - `register_backends`. It should return dictonary with names of custom backends as keys and classes, that represent those backends as values. Each class supposed to be inherited from `ckanext.datastore.backend.DatastoreBackend`.