diff --git a/examples/Basic usage - Function.ipynb b/examples/Basic usage - Function.ipynb index c1cb77f9..73a60359 100644 --- a/examples/Basic usage - Function.ipynb +++ b/examples/Basic usage - Function.ipynb @@ -4,14 +4,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Basic FISSA usage" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "This notebook contains a step-by-step example of how to use the function-based interface to the [FISSA](https://github.com/rochefort-lab/fissa) toolbox.\n", + "# Basic FISSA usage\n", + "\n", + "This notebook contains a step-by-step example of how to use the function-based high-level interface to the [FISSA](https://github.com/rochefort-lab/fissa) toolbox, [fissa.run_fissa](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.run_fissa).\n", "\n", "For more details about the methodology behind FISSA, please see our paper:\n", "\n", @@ -43,7 +38,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We also import some plotting dependencies which we'll make use in this notebook to display the results." + "We also need to import some plotting dependencies which we'll make use in this notebook to display the results." ] }, { @@ -73,9 +68,9 @@ "source": [ "## Running FISSA\n", "\n", - "With the function-based interface to FISSA, everything is handled in a single function call to ``fissa.run_fissa``, which returns the decontaminated signals.\n", + "With the function-based interface to FISSA, everything is handled in a single function call to [fissa.run_fissa](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.run_fissa). The function takes as its input is the raw signals, and returns the decontaminated signals.\n", "\n", - "The mandatory inputs to `fissa.run_fissa` are:\n", + "The mandatory inputs to [fissa.run_fissa](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.run_fissa) are:\n", "\n", "- the experiment images\n", "- the regions of interest (ROIs) to extract\n", @@ -84,7 +79,8 @@ "```python\n", "images = \"folder\"\n", "```\n", - "Each of these tiff stacks in the folder (e.g. `\"folder/trial_001.tif\"`) is a trial with many frames.\n", + "Each of these tiff-stacks in the folder (e.g. `\"folder/trial_001.tif\"`) is a trial with many frames.\n", + "Although we refer to one trial as an `image`, it is actually a video recording.\n", "\n", "Alternatively, the image data can be given as a list of paths to tiffs:\n", "```python\n", @@ -95,20 +91,38 @@ "images = [array1, array2, array3, ...]\n", "```\n", "\n", - "For the ROIs input, you can either provide a single set of ROIs, or a set of ROIs for every image.\n", + "For the regions of interest (ROIs) input, you can either provide a single set of ROIs, or a set of ROIs for every image.\n", "\n", "If the ROIs were defined using ImageJ, use ImageJ's export function to save them in a zip.\n", - "Then, provide the ROI filename,\n", + "Then, provide the ROI filename.\n", "```python\n", "rois = \"rois.zip\" # for a single set of ROIs used across all images\n", "```\n", - "or list of ROI filenames\n", + "The same set of ROIs will be used for every image in `images`.\n", + "\n", + "Sometimes there is motion between trials causing the alignment of the ROIs to drift.\n", + "In such a situation, you may need to use a slightly different location of the ROIs for each trial.\n", + "This can be handled by providing FISSA with a list of ROI sets — one ROI set (i.e. one ImageJ zip file) per trial.\n", "```python\n", "rois = [\"rois1.zip\", \"rois2.zip\", ...] # for a unique roiset for each image\n", "```\n", - "Defining a different roiset per image can be useful if you need to adjust for motion drift, for example.\n", + "Please note that the ROIs defined in each ROI set must correspond to the same physical reigons across all trials, and that the order must be consistent.\n", + "That is to say, the 1st ROI listed in each ROI set must correspond to the same item appearing in each trial, etc.\n", "\n", - "Then, we can run FISSA as follows." + "In this notebook, we will demonstrate how to use FISSA with ImageJ ROI sets, saved as zip files.\n", + "However, you are not restricted to providing your ROIs to FISSA in this format.\n", + "FISSA will also accept ROIs which are arbitrarily defined by providing them as arrays (`numpy.ndarray` objects).\n", + "ROIs provided in this way can be defined either as boolean-valued masks indicating the presence of a ROI per-pixel in the image, or defined as a list of coordinates defining the boundary of the ROI.\n", + "For examples of such usage, see our [Suite2p](https://fissa.readthedocs.io/en/latest/examples/Suite2p%20example.html), [CNMF](https://fissa.readthedocs.io/en/latest/examples/cNMF%20example.html), and [SIMA](https://fissa.readthedocs.io/en/latest/examples/SIMA%20example.html) example notebooks." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "As an example, we will run FISSA on a small test dataset.\n", + "\n", + "The test dataset can be found and downloaded from [the examples folder of the fissa repository](https://github.com/rochefort-lab/fissa/tree/master/examples), along with the source for this example notebook." ] }, { @@ -117,7 +131,7 @@ "metadata": {}, "outputs": [], "source": [ - "# Define image and ROI locations\n", + "# Define path to imagery and to the ROI set\n", "images_location = \"exampleData/20150529\"\n", "rois_location = \"exampleData/20150429.zip\"\n", "\n", @@ -129,9 +143,10 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that although the function-based interface is very straight forward, you can only access the result which is returned by the function.\n", + "The function-based interface is very straight forward, but note that you can only access the result which is returned by the function.\n", "\n", - "If you need to access the raw traces, ROI masks, or demixing matrix, you need to use the more flexible [object-oriented (class based) interface](https://rochefort-lab.github.io/fissa/examples/Basic%20usage.html), with [fissa.Experiment](https://fissa.readthedocs.io/en/stable/source/packages/fissa.core.html#fissa.core.Experiment), instead." + "If you need to access the raw traces, ROI masks, or demixing matrix, you need to use the more flexible object-oriented (class based) interface using [fissa.Experiment](https://fissa.readthedocs.io/en/stable/source/packages/fissa.core.html#fissa.core.Experiment) instead.\n", + "An example of this is given in our [object-oriented example usage notebook](https://rochefort-lab.github.io/fissa/examples/Basic%20usage.html)." ] }, { @@ -140,7 +155,7 @@ "source": [ "## Working with results\n", "\n", - "The output of `fissa.run_fissa()` is structured as a 2-d array of 2-d arrays (it can't be a 4-d array because of differing trial lengths).\n", + "The output of ``fissa.run_fissa`` is structured as a 2-d array of 2-d arrays (it can't be a 4-d array because of trials generally don't have the same number of frames).\n", "\n", "The results from the cell (ROI) numbered `c` and the trial (TIFF) numbered `t` are located at `result[c, t][0, :]`.\n", "\n", @@ -258,8 +273,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Comparing ROI signal to neuripil region signals\n", - "It can be very instructive to compare the signal in the central ROI with the surrounding neuropil regions. These can be found for cell `c` and trial `t` in `raw[c][t][i,:]`, with `i=0` being the cell, and `i=1,2,3,...` indicating the surrounding regions.\n", + "### Comparing ROI signal to neuropil region signals\n", + "It can be very instructive to compare the signal in the central ROI with the surrounding neuropil regions. These can be found for cell `c` and trial `t` in `raw[c, t][i, :]`, with `i=0` being the cell, and `i=1,2,3,...` indicating the surrounding regions.\n", "\n", "Below we compare directly the raw ROI trace, the decontaminated trace, and the surrounding neuropil region traces." ] @@ -270,8 +285,10 @@ "metadata": {}, "outputs": [], "source": [ - "# get the total number of regions\n", - "nRegions = raw[0][0].shape[0] - 1\n", + "# Get the number of neuropil/surrounding regions.\n", + "# The raw data has the raw ROI signal in raw[:, :][0] and raw surround\n", + "# signals in the rest of the 3rd dimension.\n", + "nRegions = raw[0, 0].shape[0] - 1\n", "\n", "# Select the ROI and trial to plot\n", "roi = 2\n", @@ -335,12 +352,12 @@ "\n", "The default output from `fissa.run_fissa` is in the same units as the raw input (candelas per unit area).\n", "\n", - "It is often desired to calculate the intensity of a signal relative to the baseline value, df/f0, for the traces.\n", + "It is often desirable to calculate the intensity of a signal relative to the baseline value, df/f0, for the traces.\n", "`fissa.run_fissa` will do this for you provide the argument `return_deltaf=True`, and the sampling frequency of your TIFF files with `freq=sample_frequency`.\n", "The sampling frequency must be provided because the data is smoothed in order to determine the baseline value f0.\n", "\n", "When `return_deltaf=True`, `run_fissa` will return the df/f0 output *instead* of the source signal traces scaled at the recording intensity.\n", - "If you need to access both the standard FISSA output *and* the df/f0 output at the same time, you should use the more flexible [fissa.Experiment](https://fissa.readthedocs.io/en/stable/source/packages/fissa.core.html#fissa.core.Experiment) FISSA interface instead, as described in [this example](https://rochefort-lab.github.io/fissa/examples/Basic%20usage.html)." + "If you need to access both the standard FISSA output *and* the df/f0 output at the same time, you need to use the more flexible [fissa.Experiment](https://fissa.readthedocs.io/en/stable/source/packages/fissa.core.html#fissa.core.Experiment) FISSA interface instead, as described in [this example](https://fissa.readthedocs.io/en/latest/examples/Basic%20usage.html#df/f0)." ] }, { @@ -360,7 +377,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Note that by default, f0 is determined as the minimum across all trials (all tiffs) to ensure that results are directly comparable between trials, but you can normalise each trial individually instead if you prefer by setting `deltaf_across_trials=False`." + "Note that by default, f0 is determined as the minimum across all trials (all TIFFs) to ensure that results are directly comparable between trials, but you can normalise each trial individually instead if you prefer by providing the parameter `deltaf_across_trials=False`." ] }, { @@ -393,7 +410,7 @@ "\n", "plt.title(\"ROI {}, Trial {}\".format(roi, trial), fontsize=15)\n", "plt.xlabel(\"Time (frame number)\", fontsize=15)\n", - "plt.ylabel(\"Signal intensity (candela per unit area)\", fontsize=15)\n", + "plt.ylabel(r\"$\\Delta f\\,/\\,f_0$\", fontsize=15)\n", "plt.grid()\n", "plt.legend()\n", "plt.show()" @@ -404,7 +421,9 @@ "metadata": {}, "source": [ "Since FISSA is very good at removing contamination from the ROI signals, the minimum value on the decontaminated trace will typically be `0.`.\n", - "Consequently, we use the minimum value of the (smoothed) raw signal to provide the f0 from the raw trace for both the raw and decontaminated df/f0." + "Consequently, we use the minimum value of the (smoothed) raw signal to provide the f0 from the raw trace for both the raw and decontaminated df/f0.\n", + "\n", + "We can plot the df/f0 for every cell during every trial as follows." ] }, { @@ -417,7 +436,8 @@ "n_roi = result.shape[0]\n", "n_trial = result.shape[1]\n", "\n", - "# Find the maximum signal intensities for each ROI\n", + "# Find the maximum signal intensities for each ROI,\n", + "# so we can scale ylim the same across subplots\n", "roi_max = [\n", " np.max([np.max(result[i_roi, i_trial][0]) for i_trial in range(n_trial)])\n", " for i_roi in range(n_roi)\n", @@ -438,11 +458,7 @@ " # Labels and boiler plate\n", " plt.ylim([-0.05 * roi_max[i_roi], roi_max[i_roi] * 1.05])\n", " if i_roi == 0:\n", - " plt.ylabel(\n", - " \"Trial {}\\n\\nSignal intensity\\n(candela per unit area)\".format(\n", - " i_trial + 1\n", - " )\n", - " )\n", + " plt.ylabel(\"Trial {}\\n\\n\".format(i_trial + 1) + r\"$\\Delta f\\,/\\,f_0$\")\n", " if i_trial == 0:\n", " plt.title(\"ROI {}\".format(i_roi))\n", " if i_trial == n_trial - 1:\n", @@ -451,6 +467,13 @@ "plt.show()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For comparison purposes, we can also plot the df/f0 for the raw data against the decontaminated signal." + ] + }, { "cell_type": "code", "execution_count": null, @@ -511,9 +534,11 @@ "source": [ "## Caching\n", "\n", - "You can optionally provide FISSA with the name of a cache directory.\n", - "If this argument is given, FISSA will load any output that is present from there instead of recomputing it.\n", - "If the cache directory is empty, FISSA will save its outputs there." + "After using FISSA to decontaminate the data collected in an experiment, you will probably want to save the output for later use, so you don't have to keep re-running FISSA on the data.\n", + "\n", + "To facilitate this, an option to cache the outputs is built into FISSA.\n", + "If you provide ``fissa.run_fissa`` with an identifier to the experiment being analysed in the ``folder`` argument, it will cache results into the corresponding directory.\n", + "Later, if you call ``fissa.run_fissa`` again with the same ``folder`` argument, it will load the saved results from that cache folder instead of recomputing them." ] }, { @@ -537,6 +562,9 @@ "#\n", "# In this example, we will use the current datetime as the name of the\n", "# experiment, but you can name your experiments however you want to.\n", + "# If you want to take advantage of the caching of results, you should use\n", + "# a more descriptive name than this so you can identify the actual\n", + "# dataset that the FISSA results correspond to, and load them appropriately.\n", "\n", "import datetime\n", "\n", @@ -547,14 +575,38 @@ "print(output_folder)" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's run FISSA on this experiment again, but this time save the results to the experiment's output directory." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Run FISSA, saving to results to output_folder\n", + "result, raw = fissa.run_fissa(images_location, rois_location, folder=output_folder)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A subsequent call to ``fissa.run_fissa`` which uses the same experiment folder argument will load the cached data instead of re-running the FISSA signal separation routine from scratch." + ] + }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ - "# Run FISSA, saving to the output directory\n", - "result, raw = fissa.run_fissa(images_location, rois_location, output_folder)" + "# Run FISSA, loading results from cache in output_folder\n", + "result, raw = fissa.run_fissa(images_location, rois_location, folder=output_folder)" ] }, { @@ -563,9 +615,9 @@ "source": [ "### Exporting to MATLAB\n", "\n", - "The results can easily be exported to a MATLAB-compatible matfile as follows.\n", + "The results can easily be exported to a MATLAB-compatible [MAT-file](https://mathworks.com/help/matlab/import_export/mat-file-versions.html) as follows.\n", "\n", - "We can either use `export_to_matlab=True`, which will include export to matlab with the default filename, `'separated.mat'`, within the cache directory (the cache directory being set by our `output_folder` variable)." + "If we provide `export_to_matlab=True` to `fissa.run_fissa`, it will export the data a matfile named `\"separated.mat\"` within the cache directory (the cache directory as provided with the `folder` argument)." ] }, { @@ -575,7 +627,7 @@ "outputs": [], "source": [ "result, raw = fissa.run_fissa(\n", - " images_location, rois_location, output_folder, export_to_matlab=True\n", + " images_location, rois_location, folder=output_folder, export_to_matlab=True\n", ")" ] }, @@ -583,9 +635,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Alternatively, we can manually specify the path to the output directory.\n", - "\n", - "In this case, setting a cache directory is optional." + "Alternatively, we can export to a matfile with a custom file name by setting the `export_to_matlab` argument to the target path." ] }, { @@ -603,26 +653,29 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Loading the generated file, `output_folder/separated.mat` or your custom filename, in MATLAB will provide you with all of FISSA's outputs.\n", - "\n", - "These interface similarly as `experiment.raw`, and `experiment.result` described above, with a few small differences.\n", + "Loading the generated file (e.g. `\"output_folder/separated.mat\"`) in MATLAB will provide you with all of FISSA's outputs.\n", "\n", + "These are structured in the same way as the `raw` and `result` variables returned by `fissa.run_fissa`.\n", "With the python interface, the outputs are 2d numpy.ndarrays each element of which is itself a 2d numpy.ndarrays.\n", - "In comparison, when the output is loaded into MATLAB this becomes a 2d cell-array each element of which is a 2d matrix.\n", + "Meanwhile, when the output is loaded into MATLAB the data is structured as a 2d cell-array each element of which is a 2d matrix.\n", "\n", - "Additionally, whilst Python indexes from 0, MATLAB indexes from 1 instead.\n", + "Additionally, note that whilst Python indexes from 0, MATLAB indexes from 1 instead.\n", "As a consequence of this, the results seen on Python for a given roi and trial `experiment.result[roi, trial]` correspond to the index `S.result{roi + 1, trial + 1}` on MATLAB.\n", "\n", "Our first plot in this notebook can be replicated in MATLAB as follows:\n", "```octave\n", - "% Load the FISSA output data\n", + "%% Plot example traces in MATLAB\n", + "% Load FISSA output data in MATLAB\n", + "% ... either from the automatic file name within a cache\n", "% S = load('fissa-example/separated.mat')\n", + "% ... or from a custom output path\n", "S = load('experiment_results.mat')\n", "% Select the second trial\n", "% (On Python, this would be trial = 1)\n", "trial = 2;\n", "% Plot the result traces for each ROI\n", - "figure; hold on;\n", + "figure;\n", + "hold on;\n", "for i_roi = 1:size(S.result, 1);\n", " plot(S.result{i_roi, trial}(1, :));\n", "end\n", @@ -710,7 +763,7 @@ "# If you change the experiment parameters, you need to change the cache directory too.\n", "# Otherwise FISSA will try to reload the results from the previous run instead of\n", "# computing the new results. FISSA will throw an error if you try to load data which\n", - "# was generated with different analysis parameters to its parameters.\n", + "# was generated with different analysis parameters to the current parameters.\n", "output_folder2 = output_folder + \"_alt\"\n", "\n", "# Run FISSA with these parameters\n", @@ -768,7 +821,7 @@ "By default, FISSA loads entire TIFF files into memory at once and then manipulates all ROIs within the TIFF.\n", "This is more efficient, but can be problematic when working with very large TIFF files which are too big to be loaded into memory all at once.\n", "\n", - "If you have out-of-memory problems, you can try reducing the number of workers during the memory-intensive preparation step." + "If you run out of memory when running FISSA, you can try reducing the number of workers during the memory-intensive preparation step." ] }, { @@ -785,7 +838,8 @@ "metadata": {}, "source": [ "Alternatively, you can activate FISSA's low memory mode.\n", - "In this mode, it will load frames one at a time." + "In this mode, it will load and process frames one at a time.\n", + "This will run a fair bit slower than the default mode, but has a much lower memory requirement." ] }, { diff --git a/examples/Basic usage.ipynb b/examples/Basic usage.ipynb index 15d77ea3..deb3ec11 100644 --- a/examples/Basic usage.ipynb +++ b/examples/Basic usage.ipynb @@ -4,15 +4,12 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "# Object-oriented FISSA interface" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ + "# Object-oriented FISSA interface\n", + "\n", "This notebook contains a step-by-step example of how to use the object-oriented (class-based) interface to the [FISSA](https://github.com/rochefort-lab/fissa) toolbox.\n", "\n", + "The object-oriented interface, which involves creating a [fissa.Experiment](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.Experiment) instance, allows more flexiblity than the [fissa.run_fissa](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.run_fissa) function.\n", + "\n", "For more details about the methodology behind FISSA, please see our paper:\n", "\n", "Keemink, S. W., Lowe, S. C., Pakan, J. M. P., Dylda, E., van Rossum, M. C. W., and Rochefort, N. L. FISSA: A neuropil decontamination toolbox for calcium imaging signals, *Scientific Reports*, **8**(1):3493, 2018. doi: [10.1038/s41598-018-21640-2](https://www.doi.org/10.1038/s41598-018-21640-2).\n", @@ -43,7 +40,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We also import some plotting dependencies which we'll make use in this notebook to display the results." + "We also need to import some plotting dependencies which we'll make use in this notebook to display the results." ] }, { @@ -73,9 +70,9 @@ "source": [ "## Defining an experiment\n", "\n", - "To run a separation step with fissa, you need create a `fissa.Experiment` object, which will hold your extraction parameters and results.\n", + "To run a separation step with fissa, you need create a [fissa.Experiment](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.Experiment) object, which will hold your extraction parameters and results.\n", "\n", - "The mandatory inputs to `fissa.run_fissa` are:\n", + "The mandatory inputs to `fissa.Experiment` are:\n", "\n", "- the experiment images\n", "- the regions of interest (ROIs) to extract\n", @@ -84,7 +81,8 @@ "```python\n", "images = \"folder\"\n", "```\n", - "Each of these tiff stacks in the folder (e.g. `\"folder/trial_001.tif\"`) is a trial with many frames.\n", + "Each of these tiff-stacks in the folder (e.g. `\"folder/trial_001.tif\"`) is a trial with many frames.\n", + "Although we refer to one trial as an `image`, it is actually a video recording.\n", "\n", "Alternatively, the image data can be given as a list of paths to tiffs:\n", "```python\n", @@ -95,39 +93,38 @@ "images = [array1, array2, array3, ...]\n", "```\n", "\n", - "For the ROIs input, you can either provide a single set of ROIs, or a set of ROIs for every image.\n", + "For the regions of interest (ROIs) input, you can either provide a single set of ROIs, or a set of ROIs for every image.\n", "\n", "If the ROIs were defined using ImageJ, use ImageJ's export function to save them in a zip.\n", - "Then, provide the ROI filename,\n", + "Then, provide the ROI filename.\n", "```python\n", "rois = \"rois.zip\" # for a single set of ROIs used across all images\n", "```\n", - "or list of ROI filenames\n", + "The same set of ROIs will be used for every image in `images`.\n", + "\n", + "Sometimes there is motion between trials causing the alignment of the ROIs to drift.\n", + "In such a situation, you may need to use a slightly different location of the ROIs for each trial.\n", + "This can be handled by providing FISSA with a list of ROI sets — one ROI set (i.e. one ImageJ zip file) per trial.\n", "```python\n", "rois = [\"rois1.zip\", \"rois2.zip\", ...] # for a unique roiset for each image\n", "```\n", - "Defining a different roiset per image can be useful if you need to adjust for motion drift, for example.\n", - "\n", - "Then, we can define our experiment as follows." + "Please note that the ROIs defined in each ROI set must correspond to the same physical reigons across all trials, and that the order must be consistent.\n", + "That is to say, the 1st ROI listed in each ROI set must correspond to the same item appearing in each trial, etc.\n", + "\n", + "In this notebook, we will demonstrate how to use FISSA with ImageJ ROI sets, saved as zip files.\n", + "However, you are not restricted to providing your ROIs to FISSA in this format.\n", + "FISSA will also accept ROIs which are arbitrarily defined by providing them as arrays (`numpy.ndarray` objects).\n", + "ROIs provided in this way can be defined either as boolean-valued masks indicating the presence of a ROI per-pixel in the image, or defined as a list of coordinates defining the boundary of the ROI.\n", + "For examples of such usage, see our [Suite2p](https://fissa.readthedocs.io/en/latest/examples/Suite2p%20example.html), [CNMF](https://fissa.readthedocs.io/en/latest/examples/cNMF%20example.html), and [SIMA](https://fissa.readthedocs.io/en/latest/examples/SIMA%20example.html) example notebooks." ] }, { - "cell_type": "code", - "execution_count": null, + "cell_type": "markdown", "metadata": {}, - "outputs": [], "source": [ - "# Define the folder where FISSA's outputs will be cached, so they can be\n", - "# quickly reloaded in the future without having to recompute them.\n", - "#\n", - "# This argument is optional; if it is not provided, FISSA will not save its\n", - "# results for later use.\n", - "#\n", - "# Note: you *must* use a different folder for each experiment,\n", - "# otherwise FISSA will load in the folder provided instead\n", - "# of computing results for the new experiment.\n", + "As an example, we will run FISSA on a small test dataset.\n", "\n", - "output_folder = \"fissa-example\"" + "The test dataset can be found and downloaded from [the examples folder of the fissa repository](https://github.com/rochefort-lab/fissa/tree/master/examples), along with the source for this example notebook." ] }, { @@ -136,18 +133,12 @@ "metadata": {}, "outputs": [], "source": [ - "# Define image and ROI locations\n", + "# Define path to imagery and to the ROI set\n", "images_location = \"exampleData/20150529\"\n", "rois_location = \"exampleData/20150429.zip\"\n", "\n", - "experiment = fissa.Experiment(images_location, rois_location, output_folder)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Previously analyzed experiments in output_folder will be loaded, if they exist, and the next step could be skipped." + "# Create the experiment object\n", + "experiment = fissa.Experiment(images_location, rois_location)" ] }, { @@ -155,7 +146,9 @@ "metadata": {}, "source": [ "### Extracting traces and separating them\n", - "Next, we need to extract the traces and separate them:" + "\n", + "Now we have our experiment object, we need to call the [separate()](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.Experiment.separate) method to run FISSA on the data.\n", + "FISSA will extract the traces, and then separate them." ] }, { @@ -167,24 +160,13 @@ "experiment.separate()" ] }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to redo preparation and/or separation you can set:\n", - "```\n", - "experiment.separate(redo_prep=True, redo_sep=True)\n", - "```\n", - "(If you redo prepartion this will also redo the separation, to make sure these always match up)." - ] - }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Accessing results\n", "\n", - "After running ```experiment.separate()``` the results are stored as follows." + "After running [experiment.separate()](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.Experiment.separate) the analysis parameters, raw traces, output signals, ROI definitions, and mean images are stored as attributes of the experiment object, and can be accessed as follows." ] }, { @@ -193,7 +175,9 @@ "source": [ "### Mean image\n", "\n", - "You can get the temporal-mean image for a trial with `experiment.means`, which we plot below." + "The temporal-mean image for each trial is stored in `experiment.means`.\n", + "\n", + "We can read out and plot the mean of one of the trials as follows." ] }, { @@ -210,6 +194,16 @@ "plt.show()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Plotting the mean image for each trial can be useful to see if there is motion drift between trials.\n", + "\n", + "As a summary, you can also take the mean over all trials.\n", + "Some cells don't appear in every trial, so the overall mean may indicate the location of more cells than the mean image from a single trial." + ] + }, { "cell_type": "code", "execution_count": null, @@ -229,17 +223,19 @@ "source": [ "### ROI outlines\n", "\n", - "The ROI outlines, as well as the extra neuropil regions, can be found as in ```experiment.roi_polys``` as follows. For cell number ```c``` and tiff number `t`, the set of ROIs for that cell and tiff is at\n", + "The ROI outlines, and the definitions of the surrounding neuropil regions added by FISSA to determine the contaminating signals, are stored in the ``experiment.roi_polys`` attribute.\n", + "For cell number ```c``` and TIFF number `t`, the set of ROIs for that cell and TIFF is located at\n", "```python\n", - "experiment.roi_polys[c][t][0][0] # basic ROI\n", - "experiment.roi_polys[c][t][n][0] # n = 1, 2, 3, ... the neuropil regions\n", + "experiment.roi_polys[c, t][0][0] # user-provided ROI, converted to polygon format\n", + "experiment.roi_polys[c, t][n][0] # n = 1, 2, 3, ... the neuropil regions\n", "```\n", - "Sometimes ROIs cannot be expressed as a single polygon (e.g. a ring-ROI), in those cases several polygons are used to describe it as:\n", + "\n", + "Sometimes ROIs cannot be expressed as a single polygon (e.g. a ring-ROI, which needs a line for the outside and a line for the inside); in those cases several polygons are used to describe it as:\n", "```python\n", - "experiment.roi_polys[c][t][n][i] # i iterates over the different polygons\n", + "experiment.roi_polys[c, t][n][i] # i iterates over the series of polygons defining the ROI\n", "```\n", "\n", - "As an example, plotting the first region of interest plus its surrounding neuropil subregions, overlaid on top of the mean image for one trial." + "As an example, we will plot the first ROI along with its surrounding neuropil subregions, overlaid on top of the mean image for one trial." ] }, { @@ -257,11 +253,11 @@ "# Plot the mean image for the trial\n", "plt.figure(figsize=(7, 7))\n", "plt.imshow(experiment.means[trial], cmap=\"gray\")\n", - "# Get axes limits\n", + "# Get current axes limits\n", "XLIM = plt.xlim()\n", "YLIM = plt.ylim()\n", "\n", - "# Check the number of neuropil\n", + "# Check the number of neuropil regions\n", "n_npil = len(experiment.roi_polys[roi, trial]) - 1\n", "\n", "# Plot all the neuropil regions in yellow\n", @@ -285,7 +281,7 @@ " alpha=0.6,\n", " )\n", "\n", - "# Reset axes limits\n", + "# Reset axes limits to be correct for the image\n", "plt.xlim(XLIM)\n", "plt.ylim(YLIM)\n", "\n", @@ -293,6 +289,13 @@ "plt.show()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Similarly, we can plot the location of all 4 ROIs used in this experiment." + ] + }, { "cell_type": "code", "execution_count": null, @@ -327,16 +330,17 @@ "source": [ "### FISSA extracted traces\n", "\n", - "The final extracted traces can be found in ```experiment.result``` as follows. For cell number ```c``` and tiff number `t`, the final extracted trace is given by:\n", + "The final signals after separation can be found in ``experiment.result`` as follows.\n", + "For cell number ``c`` and TIFF number ``t``, the extracted trace is given by:\n", "```python\n", - "experiment.result[c][t][0, :]\n", + "experiment.result[c, t][0, :]\n", "```\n", "\n", - "In `experiment.result` one can find the signals present in the cell ROI, ordered by how strongly they are present (relative to the surrounding regions). `experiment.result[c][t][0, :]` gives the most strongly present signal, and is considered the cell's \"true\" signal. `[i, :]` for `i=1,2,3,...` gives the other signals which are present in the ROI, but driven by other cells or neuropil.\n", + "In ``experiment.result`` one can find the signals present in the cell ROI, ordered by how strongly they are present (relative to the surrounding regions). ``experiment.result[c, t][0, :]`` gives the most strongly present signal, and is considered the cell's \"true\" signal. ``[i, :]`` for ``i=1,2,3,...`` gives the other signals which are present in the ROI, but driven by other cells or neuropil.\n", "\n", "### Before decontamination\n", "\n", - "The raw extracted signals can be found in `experiment.raw` in the same way. Now in `experiment.raw[c][t][i,:]`, `i` indicates the region number, with `i=0` being the cell, and `i=1,2,3,...` indicating the surrounding regions.\n", + "The raw extracted signals can be found in ``experiment.raw`` in the same way. Now in ``experiment.raw[c, t][i, :]``, ``i`` indicates the region number, with ``i=0`` being the cell, and ``i=1,2,3,...`` indicating the surrounding regions.\n", "\n", "As an example, plotting the raw and extracted signals for the second trial for the third cell:" ] @@ -455,8 +459,8 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "### Comparing ROI signal to neuripil region signals\n", - "It can be very instructive to compare the signal in the central ROI with the surrounding neuropil regions. These can be found for cell `c` and trial `t` in `experiment.raw[c][t][i,:]`, with `i=0` being the cell, and `i=1,2,3,...` indicating the surrounding regions.\n", + "### Comparing ROI signal to neuropil region signals\n", + "It can be very instructive to compare the signal in the central ROI with the surrounding neuropil regions. These can be found for cell `c` and trial `t` in `experiment.raw[c, t][i, :]`, with `i=0` being the cell, and `i=1,2,3,...` indicating the surrounding regions.\n", "\n", "Below we compare directly the raw ROI trace, the decontaminated trace, and the surrounding neuropil region traces." ] @@ -467,7 +471,7 @@ "metadata": {}, "outputs": [], "source": [ - "# get the total number of regions\n", + "# Get the total number of regions\n", "nRegions = experiment.nRegions\n", "\n", "# Select the ROI and trial to plot\n", @@ -536,7 +540,7 @@ "### df/f0\n", "\n", "It is often useful to calculate the intensity of a signal relative to the baseline value, df/f0, for the traces.\n", - "This can be done with FISSA as follows." + "This can be done with FISSA by calling the [experiment.calc_deltaf](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.Experiment.calc_deltaf) method as follows." ] }, { @@ -545,18 +549,22 @@ "metadata": {}, "outputs": [], "source": [ - "sample_frequency = 10 # Hz\n", + "sampling_frequency = 10 # Hz\n", "\n", - "experiment.calc_deltaf(freq=sample_frequency, across_trials=True)" + "experiment.calc_deltaf(freq=sampling_frequency)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ - "Note that by default, f0 is determined as the minimum across all trials (all tiffs) to ensure that results are directly comparable between trials, but you can normalise each trial individually instead if you prefer by setting `across_trials=False`.\n", + "The sampling frequency is required because we our process for determining f0 involves applying a lowpass filter to the data.\n", + "\n", + "Note that by default, f0 is determined as the minimum across all trials (all TIFFs) to ensure that results are directly comparable between trials, but you can normalise each trial individually instead if you prefer by providing the parameter ``across_trials=False``.\n", + "\n", + "Since FISSA is very good at removing contamination from the ROI signals, the minimum value on the decontaminated trace will typically be ``0.``. Consequently, we use the minimum value of the (smoothed) raw signal to provide the f0 from the raw trace for both the raw and decontaminated df/f0.\n", "\n", - "Since FISSA is very good at removing contamination from the ROI signals, the minimum value on the decontaminated trace will typically be `0.`. Consequently, we use the minimum value of the (smoothed) raw signal to provide the f0 from the raw trace for both the raw and decontaminated df/f0." + "As we performed above, we can plot the raw and decontaminated df/f0 for each ROI in each trial." ] }, { @@ -575,7 +583,7 @@ "plt.figure(figsize=(12, 6))\n", "\n", "n_frames = experiment.deltaf_result[roi, trial].shape[1]\n", - "tt = np.arange(0, n_frames, dtype=np.float64) / sample_frequency\n", + "tt = np.arange(0, n_frames, dtype=np.float64) / sampling_frequency\n", "\n", "plt.plot(\n", " tt,\n", @@ -600,6 +608,13 @@ "plt.show()" ] }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can also plot df/f0 for the raw data to compare against the decontaminated signal for each ROI and each trial." + ] + }, { "cell_type": "code", "execution_count": null, @@ -635,7 +650,7 @@ " plt.subplot(n_trial, n_roi, i_subplot)\n", " # Plot the data\n", " n_frames = experiment.deltaf_result[i_roi, i_trial].shape[1]\n", - " tt = np.arange(0, n_frames, dtype=np.float64) / sample_frequency\n", + " tt = np.arange(0, n_frames, dtype=np.float64) / sampling_frequency\n", " plt.plot(\n", " tt,\n", " experiment.deltaf_raw[i_roi][i_trial][0, :],\n", @@ -675,11 +690,106 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Exporting to MATLAB\n", + "## Caching\n", + "\n", + "After using FISSA to clean the data from an experiment, you will probably want to save the output for later use, so you don't have to keep re-running FISSA on the data all the time.\n", + "\n", + "An option to cache the results is built into FISSA.\n", + "If you provide ``fissa.run_fissa`` with the name of the experiment using the ``folder`` argument, it will cache results into that directory.\n", + "Later, if you call ``fissa.run_fissa`` again with the same experiment name (``folder`` argument), it will load the saved results from the cache instead of recomputing them." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Define the folder where FISSA's outputs will be cached, so they can be\n", + "# quickly reloaded in the future without having to recompute them.\n", + "#\n", + "# This argument is optional; if it is not provided, FISSA will not save its\n", + "# results for later use.\n", + "#\n", + "# Note: you *must* use a different folder for each experiment,\n", + "# otherwise FISSA will load in the folder provided instead\n", + "# of computing results for the new experiment.\n", + "\n", + "output_folder = \"fissa-example\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Create a new experiment object set up to save results to the specified output folder\n", + "experiment = fissa.Experiment(images_location, rois_location, folder=output_folder)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Because we have created a new experiment object, it is yet not populated with our results.\n", + "\n", + "We need to run the separate routine again to generate the outputs.\n", + "But this time, our results will be saved to the directory named ``fissa-example`` for future reference." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "experiment.separate()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Calling the separate method again, or making a new Experiment with the same experiment ``folder`` name will not have to re-run FISSA because it can use load the pre-computed results from the cache instead." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "experiment.separate()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If you need to *force* FISSA to ignore the cache and rerun the preparation and/or separation step, you can call it with ``redo_prep=True`` and/or ``redo_sep=True`` as appropriate." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "experiment.separate(redo_prep=True, redo_sep=True)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Exporting to MATLAB\n", + "\n", + "The results can easily be exported to a MATLAB-compatible [MAT-file](https://mathworks.com/help/matlab/import_export/mat-file-versions.html) by calling the [experiment.to_matfile()](https://fissa.readthedocs.io/en/latest/source/packages/fissa.core.html#fissa.core.Experiment.to_matfile) method.\n", "\n", "The results can easily be exported to a MATLAB-compatible matfile as follows.\n", "\n", - "The output will appear in the `output_folder` we supplied to `experiment` when we created it." + "The output file, ``\"separated.mat\"``, will appear in the `output_folder` we supplied to `experiment` when we created it." ] }, { @@ -695,9 +805,9 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Loading `output_folder/separated.mat` in MATLAB will give you structs for all of FISSA's outputs.\n", + "Loading the generated file (e.g. `\"output_folder/separated.mat\"`) in MATLAB will provide you with all of FISSA's outputs.\n", "\n", - "These interface similarly as `experiment.raw`, and `experiment.result` described above, with a few small differences.\n", + "These are structured similarly to `experiment.raw` and `experiment.result` described above, with a few small differences.\n", "\n", "With the python interface, the outputs are 2d numpy.ndarrays each element of which is itself a 2d numpy.ndarrays.\n", "In comparison, when the output is loaded into MATLAB this becomes a 2d cell-array each element of which is a 2d matrix.\n", @@ -707,13 +817,15 @@ "\n", "Our first plot in this notebook can be replicated in MATLAB as follows:\n", "```octave\n", + "%% Plot example traces\n", "% Load the FISSA output data\n", "S = load('fissa-example/separated.mat')\n", "% Select the third ROI, second trial\n", "% (On Python, this would be roi = 2; trial = 1;)\n", "roi = 3; trial = 2;\n", "% Plot the raw and result traces for the ROI signal\n", - "figure; hold on;\n", + "figure;\n", + "hold on;\n", "plot(S.raw{roi, trial}(1, :));\n", "plot(S.result{roi, trial}(1, :));\n", "title(sprintf('ROI %d, Trial %d', roi, trial));\n", @@ -727,10 +839,12 @@ "\n", "Assuming all ROIs are contiguous and described by a single contour, the mean image and ROI locations can be plotted in MATLAB as follows:\n", "```octave\n", + "%% Plot ROI locations overlaid on mean image\n", "% Load the FISSA output data\n", - "S = load('separated.mat')\n", + "S = load('fissa-example/separated.mat')\n", "trial = 1;\n", - "figure; hold on;\n", + "figure;\n", + "hold on;\n", "% Plot the mean image\n", "imagesc(squeeze(S.means(trial, :, :)));\n", "colormap('gray');\n", @@ -1012,16 +1126,16 @@ "However, it is also possible to extend this functionality and integrate other data formats into FISSA in order to work with other custom and/or proprietary formats that might be used in your lab.\n", "\n", "This is done by defining your own DataHandler class.\n", - "Your custom data handler should be a subclass of [fissa.extraction.DataHandlerAbstract](https://fissa.readthedocs.io/en/stable/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerAbstract), and implement the following methods:\n", + "Your custom data handler should be a subclass of [fissa.extraction.DataHandlerAbstract](https://fissa.readthedocs.io/en/latest/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerAbstract), and implement the following methods:\n", "\n", "- `image2array(image)` takes an image of whatever format and turns it into *data* (typically a numpy.ndarray).\n", "- `getmean(data)` calculates the 2D mean for a video defined by *data*.\n", "- `rois2masks(rois, data)` creates masks from the rois inputs, of appropriate size *data*.\n", "- `extracttraces(data, masks)` applies the *masks* to *data* in order to extract traces.\n", "\n", - "See [fissa.extraction.DataHandlerAbstract](https://fissa.readthedocs.io/en/stable/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerAbstract) for further description for each of the methods.\n", + "See [fissa.extraction.DataHandlerAbstract](https://fissa.readthedocs.io/en/latest/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerAbstract) for further description for each of the methods.\n", "\n", - "If you only need to handle a new *image* input format, which is converted to a numpy.ndarray, you may find it is easier to create a subclass of the default datahandler, [fissa.extraction.DataHandlerTifffile](https://fissa.readthedocs.io/en/stable/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerTifffile).\n", + "If you only need to handle a new *image* input format, which is converted to a numpy.ndarray, you may find it is easier to create a subclass of the default datahandler, [fissa.extraction.DataHandlerTifffile](https://fissa.readthedocs.io/en/latest/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerTifffile).\n", "In this case, only the `image2array` method needs to be overwritten and the other methods can be left as they are." ] }, @@ -1075,7 +1189,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "For advanced users that want to entirely replace the DataHandler with their own methods, you can also inherit a class from the abstract class, [fissa.extraction.DataHandlerAbstract](https://fissa.readthedocs.io/en/stable/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerAbstract).\n", + "For advanced users that want to entirely replace the DataHandler with their own methods, you can also inherit a class from the abstract class, [fissa.extraction.DataHandlerAbstract](https://fissa.readthedocs.io/en/latest/source/packages/fissa.extraction.html#fissa.extraction.DataHandlerAbstract).\n", "This can be useful if you want to integrate FISSA into your workflow without changing everything into the numpy array formats that FISSA usually uses internally." ] }