Merge branch 'casper-astro-soak-test-ska' of https://github.com/caspe…

…r-astro/mlib_devel into casper-astro-soak-test-ska

README.md

@@ -1,25 +1,53 @@
-# Getting Started
-## What is mlib_devel?
-
-`mlib_devel` is a set of DSP libraries and tools maintained by the [Collaboration for Astronomical Signal Processing and Electronics Research (CASPER)](https://casper.berkeley.edu). Within the collaboration, it is affectionately referred to as _The Toolflow._
-
-`mlib_devel` allows you to generate firmware designs which can run on supported Xilinx FPGA hardware platforms. It uses Xilinx ISE/Vivado to perform compiles of designs into FPGA bitcode, and MATLAB Simulink and Xilinx System Generator as a frontend to provide a graphical interface which makes it easy to design DSP pipelines. `mlib_devel` contains a suite of libraries providing common functionality needed by DSP systems used in radio astronomy -- for example: flexible FFTs, FIR filters, correlator modules, etc. Crucially, it also contains blocks providing high-level interfaces to board-level resources, such as memories, high-speed (Ethernet) IO, Analog-to-digital Converters (ADCs), and Digital-to-Analog Converters (DACs). `mlib_devel` is designed to be used with [casperfpga](https://github.com/casper-astro/casperfpga), a software suite which makes it easy to interact with firmware while it is running on an FPGA.
-
-
-The tools create an ISE/Vivado project which is compiled using a generated TCL script. The output bitstream contains more than just the bitstream and includes major design configuration and a memory map of the devices in the design accessible from software. 
-
-## JASPER mlib_devel directory structure
+# The CASPER Toolflow
 
--**jasper_library/hdl_sources**: HDL source files for all user IP (Ethernet cores, ADC interfaces, etc.)
+Welcome to the CASPER Toolflow repository, `mlib_devel`!
 
--**jasper_library/yellow_blocks**: python classes for each yellow block in the simulink xps blockset. These classes contain the python code which tells the tool flow how each yellow block should modify the project's top-level HDL source file and vivado project.
-
--**jasper_library/platforms**: a yaml file specifying information about a specific hardware platform. Mostly this is used to  map pythonic constraints - i.e., "connect signal my_led to the board's led[4] pin" - to hardware  constraints - i.e. " my_led -> LOC XXX, my_led -> IOSTD XXX". This file also includes source files the platform requires to compile. The source files in jasper_library/hdl_sources/<platform_name>/ are automatically included. jasper_library/hdl_sources/<platform_name>/top.v is used as a starting point for HDL generation.
-
-There's a few matlab scripts in **jasper_library** which turn a simulink diagram into source/configuration files that the rest of the tool flow can understand.
-
-Then there's the entire **mlib_devel/casper_library**, which is all the matlab/simulink files for the casper DSP (Digital Signal Processing) libraries.
-
-Everything in **mlib_devel/xps_base** is obsolete. It contains the pcores, which the old casper tool set utilises. Jasper does not make use of this though.
+## What is mlib_devel?
 
-Then there is the entire **mlib_devel/xps_library**, which contains all the matlab/simulink yellow block files for the casper XPS (Xilinx Platform Studio) libraries.
+The `mlib_devel` repository contains is a set of FPGA DSP libraries and programming tools developed and maintained by the [Collaboration for Astronomical Signal Processing and Electronics Research (CASPER)](http://casper.berkeley.edu/>). Within the collaboration, this collection of software is affectionately referred to as *The Toolflow.*
+
+The CASPER toolflow allows you to generate signal processing designs using MATLAB's graphical programming tool `Simulink`. These designs can be turned into FPGA bitstreams and loaded onto a variety of supported hardware platforms to perform real-time digital signal processing systems. CASPER also provides a Python software library for interacting with running designs: [casperfpga ](https://github.com/casper-astro/casperfpga).
+
+## Using mlib_devel
+
+For more information about installing and using the CASPER Toolflow, see the project's [documentation](https://casper-toolflow.readthedocs.io).
+
+CASPER also maintain a set of [tutorials](https://casper-tutorials.readthedocs.io), designed to introduce new users to the toolflow.
+
+> ***Updating an Existing Toolflow Installation***
+>
+>You can always update your installation of `mlib_devel` by pulling updated code from this repository. If you do this, chances are you'll need to update your Simulink models to match your new `mlib_devel` libraries. A script is provided to automate this process. With your model open and active, in your MATLAB prompt, run
+>```matlab
+>update_casper_blocks(bdroot)
+>```
+> This script will resynchronize every CASPER block in your design with its latest library version. Depending on the size of your model, it may take many minutes to complete!
+>As always, back up your designs before attempting such a major operation. And, if you experience problems, please riase Github issues!
+
+## Directory structure
+
+<dl>
+  <dt>casper_library</dt>
+  <dd>Simulink DSP libraries</dd>
+  <dt>xps_library</dt>
+  <dd>Simulink libraries for tool-flow supported modules (ADC interfaces, Ethernet cores, etc.)</dd>
+  <dt>xps_base</dt>
+  <dd>HDL code and Xilinx EDK wrappers used in older (ROACH2 and earlier) versions of the toolflow.</dd>
+  <dt>docs</dt>
+  <dd><a href="https://casper-toolflow.readthedocs.io">Sphinx documentation</a> for the software in this project.</dd>
+  <dt>jasper_library</dt>
+  <dd>
+    Python and MATLAB scripts required to drive the compilation process. Also platform-dependent configuration information and source-code for IP modules used by the toolflow in the following directories:
+  <dl>
+    <dt>jasper_library/platforms</dt>
+    <dd>YAML files defining the compile parameters and physical constraints of CASPER-supported FPGA platforms.</dd>
+    <dt>jasper_library/golden</dt>
+    <dd>Golden boot images for FPGA platforms which require them.</dd>
+    <dt>jasper_library/hdl_sources</dt>
+    <dd>HDL source files for all toolflow-suppled modules (eg. ADC interfaces, Ethernet cores, etc.).</dd>
+    <dt>jasper_library/sw</dt>
+    <dd>Codebase for embedded software processors used by the toolflow.</dd>
+    <dt>jasper_library/yellow_blocks</dt>
+    <dd>Python classes for each yellow block in the simulink xps_library.</dd>
+    </dl>
+  </dd>
+</dl>

docs/index.rst

@@ -1,38 +1,46 @@
 CASPER Toolflow
 =================
 
-Getting Started
-----------------
-What is mlib_devel?
-^^^^^^^^^^^^^^^^^^^^
-
-``mlib_devel`` is a set of DSP libraries and tools maintained by the `Collaboration for Astronomical Signal Processing and Electronics Research (CASPER) <http://casper-dsp.org/>`__. Within the collaboration, it is affectionately referred to as *The Toolflow.*
-
-``mlib_devel`` allows you to generate firmware designs which can run on supported Xilinx FPGA hardware platforms. It uses Xilinx ISE/Vivado to perform compiles of designs into FPGA bitcode, and MATLAB Simulink and Xilinx System Generator as a frontend to provide a graphical interface which makes it easy to design DSP pipelines. ``mlib_devel`` contains a suite of libraries providing common functionality needed by DSP systems used in radio astronomy -- for example: flexible FFTs, FIR filters, correlator modules, etc. Crucially, it also contains blocks providing high-level interfaces to board-level resources, such as memories, high-speed (Ethernet) IO, Analog-to-digital Converters (ADCs), and Digital-to-Analog Converters (DACs). ``mlib_devel`` is designed to be used with `casperfpga <https://github.com/casper-astro/casperfpga>`__, a software suite which makes it easy to interact with firmware while it is running on an FPGA.
-
-The tools create an ISE/Vivado project which is compiled using a generated TCL script. The output bitstream contains more than just the bitstream and includes major design configuration and a memory map of the devices in the design accessible from software. 
-
-JASPER mlib_devel directory structure
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-- **jasper_library/hdl_sources**: HDL source files for all user IP (Ethernet cores, ADC interfaces, etc.)
 
-- **jasper_library/yellow_blocks**: python classes for each yellow block in the simulink xps blockset. These classes contain the python code which tells the tool flow how each yellow block should modify the project's top-level HDL source file and vivado project.
-
-- **jasper_library/platforms**: a yaml file specifying information about a specific hardware platform. Mostly this is used to  map pythonic constraints - i.e., "connect signal my_led to the board's led[4] pin" - to hardware  constraints - i.e. " my_led -> LOC XXX, my_led -> IOSTD XXX". This file also includes source files the platform requires to compile. The source files in jasper_library/hdl_sources/<platform_name>/ are automatically included. jasper_library/hdl_sources/<platform_name>/top.v is used as a starting point for HDL generation.
-
-There's a few matlab scripts in **jasper_library** which turn a simulink diagram into source/configuration files that the rest of the tool flow can understand.
-
-Then there's the entire **mlib_devel/casper_library**, which is all the matlab/simulink files for the casper DSP (Digital Signal Processing) libraries.
-
-Everything in **mlib_devel/xps_base** is obsolete. It contains the pcores, which the old casper tool set utilises. Jasper does not make use of this though.
-
-Then there is the entire **mlib_devel/xps_library**, which contains all the matlab/simulink yellow block files for the casper XPS (Xilinx Platform Studio) libraries.
+What is mlib_devel?
+^^^^^^^^^^^^^^^^^^^
+
+``mlib_devel`` is a set of FPGA DSP libraries and programming tools maintained by the `Collaboration for Astronomical Signal Processing and Electronics Research (CASPER) <http://casper.berkeley.edu/>`__. Within the collaboration, it is affectionately referred to as *The Toolflow.*
+
+``mlib_devel`` allows you to generate signal processing designs using MATLAB's graphical programming tool `Simulink`. These designs can be turned into FPGA bitstreams and loaded onto a variety of supported hardware platforms to perform real-time digital signal processing systems. CASPER also provides a Python software library for interacting with running designs: `casperfpga <https://github.com/casper-astro/casperfpga>`__.
+
+
+`mlib_devel` directory structure
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+`casper_library`
+  Simulink DSP libraries
+
+`xps_library`
+  Simulink libraries for tool-flow supported modules (ADC interfaces, Ethernet cores, etc.)
+
+`xps_base`
+  HDL code and Xilinx EDK wrappers used in older (ROACH2 and earlier) versions of the toolflow.
+
+`docs`
+  `Sphinx documentation <https://casper-toolflow.readthedocs.io>`__ for the software in this project.
+`jasper_library`
+  Python and MATLAB scripts required to drive the compilation process. Also platform-dependent configuration information and source-code for IP modules used by the toolflow in the following directories.
+
+  `platforms`
+    YAML files defining the compile parameters and physical constraints of CASPER-supported FPGA platforms.
+  `golden`
+    Golden boot images for FPGA platforms which require them.
+  `hdl_sources`
+    HDL source files for all toolflow-suppled modules (eg. ADC interfaces, Ethernet cores, etc.).
+  `sw`
+    Codebase for embedded software processors used by the toolflow
+  `yellow_blocks`
+    Python classes for each yellow block in the simulink `xps_library`.
 
 Setup
 ------
 
-Depending on the hardware you are designing for you will require a different combination of the tools. The older hardware (ROACHes) use the older Xilinx software (ISE) which forces the use of different tools.
+The software stack you will require to use the toolflow will depend what hardware you are targetting. Older hardware (ROACH2 and earlier) use the older Xilinx software (ISE) which forces the use of different tools.
 
 The recommended OS is Ubuntu as it is what the majority of the collaboration are using. This makes it easier for us to support you. If you are so inclined, you could also use Red Hat, but we definitely do not support Windows. You are welcome to try but you will be on your own. You could always run Linux in a VM although this will increase your compile times.
 
@@ -72,4 +80,4 @@ Documentation
     :caption: Documentation
 
     CASPER Tutorials <http://casper-tutorials.readthedocs.io/en/latest/>
-    blockdocumentation
+    blockdocumentation

docs/requirements.txt

@@ -1,4 +1,5 @@
+markdown
 sphinx
 sphinx_rtd_theme
 recommonmark
-sphinx-markdown-tables
+sphinx-markdown-tables

jasper_library/exec_flow.py

@@ -159,7 +159,7 @@ def shell_source(script):
 
     if opts.software:
         binary = backend.binary_loc
-        output_fpg = tf.frontend_target_base[:-4] + '_%d-%02d-%02d_%02d%02d.fpg' % (
+        backend.output_fpg = tf.frontend_target_base[:-4] + '_%d-%02d-%02d_%02d%02d.fpg' % (
             tf.start_time.tm_year, tf.start_time.tm_mon, tf.start_time.tm_mday,
             tf.start_time.tm_hour, tf.start_time.tm_min)
 
@@ -177,6 +177,7 @@ def shell_source(script):
                                          backend.compile_dir)
         os.system(mkbof_cmd)
         print 'Created %s/%s' % (backend.output_dir, backend.output_bof)
-        backend.mkfpg(binary, output_fpg)
+        backend.mkfpg(binary, backend.output_fpg)
+        print 'Created %s/%s' % (backend.output_dir, backend.output_fpg)
 
 # end