Merge pull request #3159 from jessica-mitchell/device-docs

Clarify documentation related to recorders and recording backends

doc/htmldoc/devices/index.rst

@@ -3,6 +3,21 @@
 All about devices in NEST
 =========================
 
+.. grid:: 1 1 2 2
+    :gutter: 1
+
+    .. grid-item-card::  Stimulate the network
+       :class-title: sd-d-flex-row sd-align-minor-center
+       :link: stimuate_network
+       :link-type: ref
+
+    .. grid-item-card:: Get data from simulation
+       :class-title: sd-d-flex-row sd-align-minor-center
+       :link: record_simulations
+       :link-type: ref
+
+
+
 .. toctree::
   :maxdepth: 1
   :glob:

doc/htmldoc/devices/record_from_simulations.rst

@@ -38,15 +38,39 @@ Recording devices can fundamentally be subdivided into two groups:
   neuron (not the neuron to the sampler), and that the neuron must
   support the particular type of sampling.
 
+.. _recording_backends:
 
-Recorders for every-day situations
-----------------------------------
+What values can I record?
+-------------------------
+
+This depends on neuron or synapse model specified.
+
+You can get a list of properties that you can record using the ``recordables`` property.
+
+        ::
+
+         >>> nest.GetDefaults("iaf_cond_alpha")["recordables"]
+         ["g_ex", "g_in", "t_ref_remaining", "V_m"]
+
+
+Recorders available in NEST
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- :doc:`/models/multimeter`
+
+
+- :doc:`/models/spike_recorder`
+
+
+- :doc:`/models/weight_recorder`
+
+Check out the following examples to see how the recorders are used:
+
+- :doc:`/auto_examples/recording_demo`
+- :doc:`/auto_examples/multimeter_file`
+- :doc:`/auto_examples/urbanczik_synapse_example` uses all 3 recorders.
 
-- :doc:`../models/multimeter`
-- :doc:`../models/spike_recorder`
-- :doc:`../models/weight_recorder`
 
-.. _recording_backends:
 
 Where does data end up?
 -----------------------
@@ -55,10 +79,6 @@ After a recording device has collected or sampled data, the data is
 handed to a dedicated *recording backend*, set for each recorder.
 These are responsible for how the data are processed.
 
-Theoretically, recording backends are completely free in what they do
-with the data. The ones included in NEST can collect data in memory,
-display it on the terminal, or write it to files.
-
 To specify the recording backend for a given recording device, the
 property ``record_to`` of the latter has to be set to the name of the
 recording backend to be used. This can either happen already in the
@@ -67,13 +87,33 @@ call to :py:func:`.Create` or by using :py:func:`.SetStatus` on the model instan
 
 ::
 
- sr = nest.Create('spike_recorder', params={'record_to': 'ascii'})
+ sr = nest.Create("spike_recorder", params={"record_to": "memory"})
 
 Storing data in memory using the `memory` backend is the default for
 all recording devices as this does not require any additional setup of
 data paths or filesystem permissions and allows a convenient readout
 of data by the user after simulation.
 
+For example, you can use the ``events`` property to get the data output from the `memory` backend
+from any of the recorders:
+
+::
+
+   >>> spike_recorder.get("events")
+   {"senders": array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]),
+   "times": array([ 2.1,  4.4,  6.7,  8.9, 11.1, 13.3, 15.5, 17.7, 19.9, 22.1, 24.3, 26.5, 28.7])}
+
+Additional properties can be set, depending on the recorder and recording backend used.
+
+For example:
+
+::
+
+   mm = nest.Create( "multimeter",
+      params={"interval": 0.1, "record_from": ["V_m", "g_ex", "g_in"], "record_to": "ascii", "label": "my_multimeter"},
+      )
+
+
 Each recording backend may provide a specific set of parameters
 (explained in the backend documentation below) that will be included
 in the model status dictionary once the backend is set. This means
@@ -97,7 +137,7 @@ kernel attribute ``recording_backends``.
 ::
 
    >>> print(nest.recording_backends)
-   ('ascii', 'memory', 'mpi', 'screen', 'sionlib')
+   ("ascii", "memory", "mpi", "screen", "sionlib")
 
 If a recording backend has global properties (i.e., parameters shared
 by all enrolled recording devices), those can be inspected with
@@ -106,17 +146,17 @@ by all enrolled recording devices), those can be inspected with
 ::
 
    >>> nest.GetDefaults("sionlib")
-   {'buffer_size': 1024,
-    'filename': '',
-    'sion_chunksize': 262144,
-    'sion_collective': False,
-    'sion_n_files': 1}
+   {"buffer_size": 1024,
+    "filename": "",
+    "sion_chunksize": 262144,
+    "sion_collective": False,
+    "sion_n_files": 1}
 
 Such global parameters can be set using :py:func:`.SetDefaults`
 
 ::
 
-   >>> nest.SetDefaults('sionlib', {'buffer_size': 512})
+   >>> nest.SetDefaults("sionlib", {"buffer_size": 512})
 
 Built-in backends
 -----------------
@@ -126,8 +166,8 @@ NEST. Please note that the availability of some of them depends on the
 compile-time configuration for NEST. See the backend documentation for
 details.
 
-- :doc:`../models/recording_backend_memory`
-- :doc:`../models/recording_backend_ascii`
-- :doc:`../models/recording_backend_screen`
-- :doc:`../models/recording_backend_sionlib`
-- :doc:`../models/recording_backend_mpi`
+.. include:: ../models/recording_backend_memory.rst
+.. include:: ../models/recording_backend_ascii.rst
+.. include:: ../models/recording_backend_screen.rst
+.. include:: ../models/recording_backend_sionlib.rst
+.. include:: ../models/recording_backend_mpi.rst

models/multimeter.h

@@ -66,7 +66,8 @@ recordables to have them sampled during simulation.
 
 ::
 
-   mm = nest.Create('multimeter', 1, {'record_from': ['V_m', 'g_ex']})
+   mm = nest.Create('multimeter', 1, {'record_from': ['V_m', 'g_ex'],
+   'record_to': memory})
 
 The sampling interval for recordings (given in ms) can be controlled
 using the ``multimeter`` parameter ``interval``. The default value of

models/spike_recorder.h

@@ -57,13 +57,14 @@ spike creation rather than that of their arrival.
 
 ::
 
-   >>> neurons = nest.Create('iaf_psc_alpha', 5)
-   >>> sr = nest.Create('spike_recorder')
+   >>> neurons = nest.Create("iaf_psc_alpha", 5)
+   >>> sr  = nest.Create("spike_recorder", params={"record_to":"memory", "time_in_steps": False})
    >>> nest.Connect(neurons, sr)
 
-The call to ``Connect`` will fail if the connection direction is
+Note the call to ``Connect`` will fail if the connection direction is
 reversed (i.e., connecting *sr* to *neurons*).
 
+
 .. include:: ../models/recording_device.rst
 
 See also

models/weight_recorder.h

@@ -63,7 +63,7 @@ synapses that fulfill the given criteria.
 
 ::
 
-   >>> wr = nest.Create('weight_recorder')
+   >>> wr = nest.Create("weight_recorder")
    >>> nest.CopyModel("stdp_synapse", "stdp_synapse_rec", {"weight_recorder": wr})
 
    >>> pre = nest.Create("iaf_psc_alpha", 10)

nestkernel/recording_backend_ascii.h

@@ -31,10 +31,10 @@
 /* BeginUserDocs: NOINDEX
 
 Recording backend `ascii` - Write data to plain text files
-##########################################################
+----------------------------------------------------------
 
 Description
-+++++++++++
+~~~~~~~~~~~
 
 The `ascii` recording backend writes collected data persistently to a
 plain text ASCII file. It can be used for small to medium sized
@@ -77,7 +77,7 @@ for avoiding name clashes is to set the kernel attributes
 ``data_path`` or ``data_prefix``, to write to a different file.
 
 Data format
-+++++++++++
+~~~~~~~~~~~
 
 Any file written by the `ascii` recording backend starts with an
 informational header. The first header line contains the NEST version,
@@ -105,7 +105,7 @@ point offset in ms from the next integer grid point.
    controlled using the recorder property ``precision``.
 
 Parameter summary
-+++++++++++++++++
+~~~~~~~~~~~~~~~~~
 
 file_extension
     A string (default: *"dat"*) that specifies the file name extension,

nestkernel/recording_backend_memory.h

@@ -29,10 +29,10 @@
 /* BeginUserDocs: NOINDEX
 
 Recording backend `memory` - Store data in main memory
-######################################################
+------------------------------------------------------
 
 Description
-+++++++++++
+~~~~~~~~~~~
 
 When a recording device sends data to the ``memory`` backend, it is
 stored internally in efficient vectors. These vectors are made
@@ -66,7 +66,7 @@ recording device. To delete data from memory, `n_events` can be set to
 0. Other values cannot be set.
 
 Parameter summary
-+++++++++++++++++
+~~~~~~~~~~~~~~~~~
 
 events
     A dictionary containing the recorded data in the form of one numeric

nestkernel/recording_backend_mpi.h

@@ -34,10 +34,10 @@
 /* BeginUserDocs: NOINDEX
 
 Recording backend `mpi` - Send data with MPI
-############################################
+--------------------------------------------
 
 Description
-+++++++++++
+~~~~~~~~~~~
 
 .. admonition:: Availability
 
@@ -65,7 +65,7 @@ its node ID. This path can only be set outside of a `Run` context
 (i.e. after ``Prepare()`` has been called, but ``Cleanup()`` has not).
 
 Communication Protocol
-++++++++++++++++++++++
+~~~~~~~~~~~~~~~~~~~~~~
 
 The following protocol is used to exchange information between both
 MPI processes. The protocol is described using the following format
@@ -80,7 +80,7 @@ for the MPI messages: (value, number, type, source/destination, tag)
 7) ``Cleanup``  : Send at this en of the simulation (true, 1, CXX_BOOL, 0, 2)
 
 Data format
-+++++++++++
+~~~~~~~~~~~
 
 The format of the data sent is an array consisting of (id device, id node, time
 is ms).

nestkernel/recording_backend_screen.h

@@ -29,10 +29,10 @@
 /* BeginUserDocs: NOINDEX
 
 Recording backend `screen` - Write data to the terminal
-#######################################################
+-------------------------------------------------------
 
 Description
-+++++++++++
+~~~~~~~~~~~
 
 When initially conceiving and debugging simulations, it can be useful
 to check recordings in a more ad hoc fashion. The recording backend
@@ -58,13 +58,13 @@ floating point offset in ms from the next grid point.
    down the simulation.
 
 Parameter summary
-+++++++++++++++++
+~~~~~~~~~~~~~~~~~
 
 precision
    controls the number of decimal places used to write decimal numbers
    to the terminal.
 
-time_in_step
+time_in_steps
    A boolean (default: false) specifying whether to print time in
    steps, i.e., in integer multiples of the resolution and an offset,
    rather than just in ms.

nestkernel/recording_backend_sionlib.h

@@ -32,10 +32,10 @@
 /* BeginUserDocs: NOINDEX
 
 Recording backend `sionlib` - Store data to an efficient binary format
-######################################################################
+----------------------------------------------------------------------
 
 Description
-+++++++++++
+~~~~~~~~~~~
 
 .. admonition:: Availability
 
@@ -80,7 +80,7 @@ attribute. An alternative way for avoiding name clashes is to set the
 kernel attributes ``data_path`` or ``data_prefix``, to write to a different file.
 
 Data format
-+++++++++++
+~~~~~~~~~~~
 
 In contrast to other recording backends, the ``sionlib`` backend
 writes the data from all recorders using it to a single container
@@ -108,7 +108,7 @@ following figure.
    NEST SIONlib binary file format.
 
 Reading the data
-++++++++++++++++
+~~~~~~~~~~~~~~~~
 
 As the binary format of the files produced by the ``sionlib`` does not
 conform to any standard, parsing them manually might be a bit
@@ -118,15 +118,15 @@ and further documentation for this module can be found in its own
 `repository <https://github.com/nest/nest-sionlib-reader>`_.
 
 Recorder-specific parameters
-++++++++++++++++++++++++++++
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 label
     A recorder-specific string (default: *""*) that serves as alias
     name for the recording device, and which is stored in the metadata
     section of the container files.
 
 Global parameters
-+++++++++++++++++
+~~~~~~~~~~~~~~~~~
 
 These parameters can be set by assigning a nested dictionary to the
 kernel attribute ``recording_backends``. The dictionary has to have

nestkernel/recording_device.h

@@ -59,9 +59,31 @@ Data handling
 +++++++++++++
 
 All recorded data is handed over to the recording backend, selected
-via the ``record_to`` property. More details on available backends and
-their properties can be found in the :ref:`guide to recording from
-simulations <recording_backends>`.
+via the ``record_to`` property::
+
+   >>> sr  = nest.Create("spike_recorder", params={"record_to":"ascii", "time_in_steps": False})
+   >>> mm = nest.Create("multimeter", 1, {"record_from": ["V_m", "g_ex"], "record_to": "memory"})
+
+
+By default, data recorded from recorders is stored in the `memory` backend.
+You can access the data recorded by the recorders with the ``events`` property.
+
+::
+
+   mm_events = mm.get("events")
+
+.. note::
+
+   The type of recording backend you choose may affect the efficiency of your simulation.
+   The  `memory` backend is ideal for interactive work, but can only be used for limited
+   amount of data. Additionally, transferring data to disk later on may be slower than
+   directly writing from the NEST kernel via `ascii` or `sionlib` backends.
+
+   Large simulations with many threads may benefit from the `sionlib` backend, as the `ascii`
+   backend opens many files which can be very time consuming on parallel file systems.
+
+   The complete list of parameters and other recording backend options
+   can be found in the :ref:`guide to recording from simulations <recording_backends>`.
 
 Recorder properties
 +++++++++++++++++++