Started documenting codegen

dev/ideas/simple_codegen_example.py

@@ -0,0 +1,52 @@
+'''
+TODO: modify this so to just generate snippets
+'''
+from numpy import *
+from brian2 import *
+from brian2.utils.stringtools import *
+from brian2.codegen.languages.cpp_lang import *
+
+##### Define the model
+tau = 10*ms
+eqs = '''
+dV/dt = -V/tau : volt (unless-refractory)
+'''
+threshold = 'V>-50*mV'
+reset = 'V=-60*mV'
+refractory = 5*ms
+groupname = 'gp'
+N = 1000
+
+##### Generate C++ code
+
+# Use a NeuronGroup to fake the whole process
+G = NeuronGroup(N, eqs, reset=reset, threshold=threshold,
+                refractory=refractory, name=groupname,
+                codeobj_class=WeaveCodeObject,
+                )
+# Run the network for 0 seconds to generate the code
+net = Network(G)
+net.run(0*second)
+# Extract the necessary information
+ns = G.state_updater.codeobj.namespace
+code = deindent(G.state_updater.codeobj.code.main)
+arrays = []
+# Freeze all constants
+for k, v in ns.items():
+    if isinstance(v, float):
+        code = ('const double %s = %s;\n' % (k, repr(v)))+code
+    elif isinstance(v, int):
+        code = ('const int %s = %s;\n' % (k, repr(v)))+code
+    elif isinstance(v, ndarray):
+        if k.startswith('_array'):
+            dtype_spec = c_data_type(v.dtype)
+            arrays.append((k, dtype_spec, N))
+
+print '*********** DECLARATIONS **********'
+# This is just an example of what you could do with declarations, generate your
+# own code here...
+for varname, dtype_spec, N in arrays:
+    print '%s *%s = new %s [%s];' % (dtype_spec, varname, dtype_spec, N)
+
+print '*********** MAIN LOOP *************'
+print code

docs_sphinx/developer/codegen.txt

@@ -0,0 +1,182 @@
+Code generation
+~~~~~~~~~~~~~~~
+
+The following is an outline of how the Brian 2 code generation system works,
+with indicators as to which packages to look at and which bits of code to read
+for a clearer understanding.
+
+We illustrate the global process with an example, the creation and running of
+a single `NeuronGroup` object:
+
+- Parse the equations, add refractoriness to them: this isn't really part of
+  code generation.
+- Allocate memory for the state variables.
+- Create a namespace object.
+- Create `Thresholder`, `Resetter` and `StateUpdater` objects.
+  - Collect `Specifier` objects from the group and code template.
+  - Resolve the namespace, i.e. for hierarchical namespace choose just one
+    value for each variable name.
+  - Create a `CodeObject`.
+- At runtime, each object calls `CodeObject.__call__` to execute the code.
+
+Stages of code generation
+=========================
+
+Equations to abstract code
+--------------------------
+
+In the case of `Equations`, the set of equations are combined with a
+numerical integration method to generate an *abstract code block* (see below)
+which represents the integration code for a single time step.
+
+An example of this would be converting the following equations::
+
+	eqs = '''
+	dv/dt = (v0-v)/tau : volt (unless-refractory)
+	v0 : volt
+	'''
+	group = NeuronGroup(N, eqs, threshold='v>10*mV',
+	                    reset='v=0*mV', refractory=5*ms)
+
+into the following abstract code using the `exponential_euler` method (which
+is selected automatically)::
+
+	not_refractory = 1*((t - lastspike) > 0.005000)
+	_BA_v = -v0
+	_v = -_BA_v + (_BA_v + v)*exp(-dt*not_refractory/tau)
+	v = _v
+
+The code for this stage can be seen in `NeuronGroup.__init__`,
+`StateUpdater.__init__`, and `StateUpdater.update_abstract_code`
+(in ``brian2.groups.neurongroup``), and the `StateUpdateMethod` classes
+defined in the ``brian2.stateupdaters`` package.
+
+Abstract code
+-------------
+
+'Abstract code' is just a multi-line string representing a block of code which
+should be executed for each item (e.g. each neuron, each synapse). Each item
+is independent of the others in abstract code. This allows us to later
+generate code either for vectorised languages (like numpy in Python) or
+using loops (e.g. in C++).
+
+Abstract code is parsed according to Python syntax, with certain language
+constructs excluded. For example, there cannot be any conditional or looping
+statements at the moment, although support for this is in principle possible
+and may be added later. Essentially, all that is allowed at the moment is a
+sequence of arithmetical ``a = b*c`` style statements.
+
+Abstract code is provided directly by the user for threshold and reset
+statements in `NeuronGroup` and for pre/post spiking events in `Synapses`.
+
+Abstract code to snippet
+------------------------
+
+We convert abstract code into a 'snippet', which is a small segment of
+code which is syntactically correct in the target language, although it may
+not be runnable on its own (that's handled by insertion into a 'template'
+later). This is handled by the `Language` object in ``brian2.codegen.languages``.
+In the case of converting into python/numpy code this typically doesn't involve
+any changes to the code at all because the original code is in Python
+syntax. For conversion to C++, we have to do some syntactic transformations
+(e.g. ``a**b`` is converted to ``pow(a, b)``), and add declarations for
+certain variables (e.g. converting ``x=y*z`` into ``const double x = y*z;``).
+
+An example of a snippet in C++ for the equations above::
+
+	const double v0 = _ptr_array_neurongroup_v0[_neuron_idx];
+	const double lastspike = _ptr_array_neurongroup_lastspike[_neuron_idx];
+	bool not_refractory = _ptr_array_neurongroup_not_refractory[_neuron_idx];
+	double v = _ptr_array_neurongroup_v[_neuron_idx];
+	not_refractory = 1 * (t - lastspike > 0.0050000000000000001);
+	const double _BA_v = -(v0);
+	const double _v = -(_BA_v) + (_BA_v + v) * exp(-(dt) * not_refractory / tau);
+	v = _v;
+	_ptr_array_neurongroup_not_refractory[_neuron_idx] = not_refractory;
+	_ptr_array_neurongroup_v[_neuron_idx] = v;
+
+The code path that includes snippet generation will be discussed in more detail
+below, since it involves the concepts of namespaces and specifiers which we
+haven't covered yet.
+
+Snippet to code block
+---------------------
+
+The final stage in the generation of a runnable code block is the insertion
+of a snippet into a template. These use the Jinja2 template specification
+language. This is handled in ``brian2.codegen.templates``.
+
+An example of a template for Python thresholding::
+
+	# USE_SPECIFIERS { not_refractory, lastspike, t }	
+	{% for line in code_lines %}
+	{{line}}
+	{% endfor %}
+	_return_values, = _cond.nonzero()
+	# Set the neuron to refractory
+	not_refractory[_return_values] = False
+	lastspike[_return_values] = t
+
+and the output code from the example equations above::
+
+	# USE_SPECIFIERS { not_refractory, lastspike, t }	
+	v = _array_neurongroup_v
+	_cond = v > 10 * mV
+	_return_values, = _cond.nonzero()
+	# Set the neuron to refractory
+	not_refractory[_return_values] = False
+	lastspike[_return_values] = t
+
+Code block to executing code
+----------------------------
+
+A code block represents runnable code. Brian operates in two different regimes,
+either in runtime or standalone mode. In runtime mode, memory allocation and
+overall simulation control is handled by Python and numpy, and code objects
+operate on this memory when called directly by Brian. This is the typical
+way that Brian is used, and it allows for a rapid development cycle. However,
+we also support a standalone mode in which an entire project workspace is
+generated for a target language or device by Brian, which can then be
+compiled and run independently of Brian. Each mode has different templates,
+and does different things with the outputted code blocks. For runtime mode,
+in Python/numpy code is executed by simply calling the ``exec`` statement
+on the code block in a given namespace. For C++/weave code, the
+``scipy.weave.inline`` function is used. In standalone mode, the templates
+will typically each be saved into different files.
+
+Key concepts
+============
+
+Namespaces
+----------
+
+In general, a namespace is simply a mapping/dict from names to values. In Brian
+we use the term 'namespace' in two ways. The high level `CompoundNamespace`
+object in ``brian2.core.namespace`` allows the definition of a nested
+hierarchy of named namespaces. The final namespace that code is executed in is a
+simple Python dictionary mapping names to values. Before that final namespace
+is generated, it goes through the process of 'namespace resolution'.
+
+TODO: Namespace resolution.
+
+Specifiers
+----------
+
+Templates
+---------
+
+Code guide
+==========
+
+This section includes a guide to the various relevant packages and subpackages
+involved in the code generation process.
+
+	codegen
+	  codegen.functions
+	  codegen.languages
+	  codegen.runtime
+	core.specifiers
+	equations
+	groups
+	parsing
+	stateupdaters

docs_sphinx/developer/index.rst

@@ -16,3 +16,4 @@ interest to others wishing to better understand how Brian works internally.
    state_update
    new_magic_and_clocks
    preferences
+   codegen

examples/I-F_curve_LIF.py

@@ -7,7 +7,7 @@
 '''
 from pylab import *
 from brian2 import *
-
+BrianLogger.log_level_debug()
 #brian_prefs.codegen.target = 'weave'
 
 N = 1000