examples/pybids_tutorial.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to `pybids`\n",
    "\n",
    "[`pybids`](https://github.com/bids-standard/pybids) is a tool to query, summarize and manipulate data using the BIDS standard. \n",
    "In this tutorial we will use a `pybids` test dataset to illustrate some of the functionality of `pybids.layout`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:23.628184Z",
     "start_time": "2018-08-01T20:05:22.060937Z"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from bids import BIDSLayout\n",
    "from bids.tests import get_test_data_path\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `BIDSLayout`\n",
    "\n",
    "At the core of pybids is the `BIDSLayout` object. A `BIDSLayout` is a lightweight Python class that represents a BIDS project file tree and provides a variety of helpful methods for querying and manipulating BIDS files. While the `BIDSLayout` initializer has a large number of arguments you can use to control the way files are indexed and accessed, you will most commonly initialize a `BIDSLayout` by passing in the BIDS dataset root location as a single argument:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:23.739377Z",
     "start_time": "2018-08-01T20:05:23.630505Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BIDS Layout: .../pybids/bids/tests/data/7t_trt | Subjects: 10 | Sessions: 20 | Runs: 20"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here we're using an example BIDS dataset that's bundled with the pybids tests\n",
    "data_path = os.path.join(get_test_data_path(), '7t_trt')\n",
    "\n",
    "# Initialize the layout\n",
    "layout = BIDSLayout(data_path)\n",
    "\n",
    "# Print some basic information about the layout\n",
    "layout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Querying the `BIDSLayout`\n",
    "When we initialize a `BIDSLayout`, all of the files and metadata found under the specified root folder are indexed. This can take a few seconds (or, for very large datasets, a minute or two). Once initialization is complete, we can start querying the `BIDSLayout` in various ways. The workhorse method is [`.get()`](https://bids-standard.github.io/pybids/generated/bids.grabbids.BIDSLayout.html#bids.grabbids.BIDSLayout.get). If we call `.get()` with no additional arguments, we get back a list of all the BIDS files in our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are 339 files in the layout.\n",
      "\n",
      "The first 10 files are:\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<BIDSJSONFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/dataset_description.json'>,\n",
       " <BIDSDataFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/participants.tsv'>,\n",
       " <BIDSFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/README'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/anat/sub-01_ses-1_T1map.nii.gz'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/anat/sub-01_ses-1_T1w.nii.gz'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_magnitude1.nii.gz'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_magnitude2.nii.gz'>,\n",
       " <BIDSJSONFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_phasediff.json'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_phasediff.nii.gz'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-2_magnitude1.nii.gz'>]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_files = layout.get()\n",
    "print(\"There are {} files in the layout.\".format(len(all_files)))\n",
    "print(\"\\nThe first 10 files are:\")\n",
    "all_files[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The returned object is a Python list. By default, each element in the list is a `BIDSFile` object. We discuss the `BIDSFile` object in much more detail below. For now, let's simplify things and work with just filenames:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/dataset_description.json',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/participants.tsv',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/README',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/anat/sub-01_ses-1_T1map.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/anat/sub-01_ses-1_T1w.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_magnitude1.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_magnitude2.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_phasediff.json',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-1_phasediff.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/fmap/sub-01_ses-1_run-2_magnitude1.nii.gz']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.get(return_type='filename')[:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, we get back only the names of the files.\n",
    "\n",
    "### Filtering files by entities\n",
    "The utility of the `BIDSLayout` would be pretty limited if all we could do was retrieve a list of all files in the dataset. Fortunately, the `.get()` method accepts all kinds of arguments that allow us to filter the result set based on specified criteria. In fact, we can pass *any* BIDS-defined keywords (or, as they're called in PyBIDS, *entities*) as constraints. For example, here's how we would retrieve all BOLD runs with `.nii.gz` extensions for subject `'01'`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/func/sub-01_ses-1_task-rest_acq-fullbrain_run-1_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/func/sub-01_ses-1_task-rest_acq-fullbrain_run-2_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/func/sub-01_ses-1_task-rest_acq-prefrontal_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-2/func/sub-01_ses-2_task-rest_acq-fullbrain_run-1_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-2/func/sub-01_ses-2_task-rest_acq-fullbrain_run-2_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-2/func/sub-01_ses-2_task-rest_acq-prefrontal_bold.nii.gz']"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Retrieve filenames of all BOLD runs for subject 01\n",
    "layout.get(subject='01', extension='nii.gz', suffix='bold', return_type='filename')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you're wondering what entities you can pass in as filtering arguments, the answer is contained in the `.json` configuration files [housed here](https://github.com/bids-standard/pybids/tree/master/bids/layout/config). To save you the trouble, here are a few of the most common entities:\n",
    "\n",
    "* `suffix`: The part of a BIDS filename just before the extension (e.g., `'bold'`, `'events'`, `'physio'`, etc.).\n",
    "* `subject`: The subject label\n",
    "* `session`: The session label\n",
    "* `run`: The run index\n",
    "* `task`: The task name\n",
    "\n",
    "New entities are continually being defined as the spec grows, and in principle (though not always in practice), PyBIDS should be aware of all entities that are defined in the BIDS specification.\n",
    "\n",
    "### Filtering by metadata\n",
    "All of the entities listed above are found in the names of BIDS files. But sometimes we want to search for files based not just on their names, but also based on metadata defined (per the BIDS spec) in JSON files. Fortunately for us, when we initialize a `BIDSLayout`, all metadata files associated with BIDS files are automatically indexed. This means we can pass any key that occurs in any JSON file in our project as an argument to `.get()`. We can combine these with any number of core BIDS entities (like `subject`, `run`, etc.).\n",
    "\n",
    "For example, say we want to retrieve all files where (a) the value of `SamplingFrequency` (a metadata key) is `100`, (b) the `acquisition` type is `'prefrontal'`, and (c) the subject is `'01'` or `'02'`. Here's how we can do that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<BIDSDataFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/func/sub-01_ses-1_task-rest_acq-prefrontal_physio.tsv.gz'>,\n",
       " <BIDSDataFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-2/func/sub-01_ses-2_task-rest_acq-prefrontal_physio.tsv.gz'>,\n",
       " <BIDSDataFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-02/ses-1/func/sub-02_ses-1_task-rest_acq-prefrontal_physio.tsv.gz'>,\n",
       " <BIDSDataFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-02/ses-2/func/sub-02_ses-2_task-rest_acq-prefrontal_physio.tsv.gz'>]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Retrieve all files where SamplingFrequency (a metadata key) = 100\n",
    "# and acquisition = prefrontal, for the first two subjects\n",
    "layout.get(subject=['01', '02'], SamplingFrequency=100, acquisition=\"prefrontal\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice that we passed a list in for `subject` rather than just a string. This principle applies to all filters: you can always pass in a list instead of a single value, and this will be interpreted as a logical disjunction (i.e., a file must match any one of the provided values).\n",
    "\n",
    "### Other `return_type` values\n",
    "While we'll typically want to work with either `BIDSFile` objects or filenames, we can also ask `get()` to return unique values (or ids) of particular entities. For example, say we want to know which subjects have at least one `T1w` file. We can request that information by setting `return_type='id'`. When using this option, we also need to specify a target entity (or metadata keyword) called `target`. This combination tells the `BIDSLayout` to return the unique values for the specified `target` entity. For example, in the next example, we ask for all of the unique subject IDs that have at least one file with a `T1w` suffix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:23.964766Z",
     "start_time": "2018-08-01T20:05:23.957477Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['01', '02', '03', '04', '05', '06', '07', '08', '09', '10']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Ask get() to return the ids of subjects that have T1w files\n",
    "layout.get(return_type='id', target='subject', suffix='T1w')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If our `target` is a BIDS entity that corresponds to a particular directory in the BIDS spec (e.g., `subject` or `session`) we can also use `return_type='dir'` to get all matching subdirectories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:23.983125Z",
     "start_time": "2018-08-01T20:05:23.975085Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-02',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-03',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-04',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-05',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-06',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-07',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-08',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-09',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-10']"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.get(return_type='dir', target='subject')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Other `get()` options\n",
    "The `.get()` method has a number of other useful arguments that control its behavior. We won't discuss these in detail here, but briefly, here are a couple worth knowing about:\n",
    "* `regex_search`: If you set this to `True`, string filter argument values will be interpreted as regular expressions.\n",
    "* `scope`: If your BIDS dataset contains BIDS-derivatives sub-datasets, you can specify the scope (e.g., `derivatives`, or a BIDS-Derivatives pipeline name) of the search space."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The `BIDSFile`\n",
    "When you call `.get()` on a `BIDSLayout`, the default returned values are objects of class `BIDSFile`. A `BIDSFile` is a lightweight container for individual files in a BIDS dataset. It provides easy access to a variety of useful attributes and methods. Let's take a closer look. First, let's pick a random file from our existing `layout`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:23.926278Z",
     "start_time": "2018-08-01T20:05:23.741189Z"
    },
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<BIDSDataFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/func/sub-01_ses-1_task-rest_acq-fullbrain_run-1_physio.tsv.gz'>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pick the 15th file in the dataset\n",
    "bf = layout.get()[15]\n",
    "\n",
    "# Print it\n",
    "bf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are some of the attributes and methods available to us in a `BIDSFile` (note that some of these are only available for certain subclasses of `BIDSFile`; e.g., you can't call `get_image()` on a `BIDSFile` that doesn't correspond to an image file!):\n",
    "* `.path`: The full path of the associated file\n",
    "* `.filename`: The associated file's filename (without directory)\n",
    "* `.dirname`: The directory containing the file\n",
    "* `.get_entities()`: Returns information about entities associated with this `BIDSFile` (optionally including metadata)\n",
    "* `.get_image()`: Returns the file contents as a nibabel image (only works for image files)\n",
    "* `.get_df()`: Get file contents as a pandas DataFrame (only works for TSV files)\n",
    "* `.get_metadata()`: Returns a dictionary of all metadata found in associated JSON files\n",
    "* `.get_associations()`: Returns a list of all files associated with this one in some way\n",
    "\n",
    "Let's see some of these in action."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'acquisition': 'fullbrain',\n",
       " 'datatype': 'func',\n",
       " 'extension': 'tsv.gz',\n",
       " 'run': 1,\n",
       " 'session': '1',\n",
       " 'subject': '01',\n",
       " 'suffix': 'physio',\n",
       " 'task': 'rest'}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print all the entities associated with this file, and their values\n",
    "bf.get_entities()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Columns': ['cardiac', 'respiratory', 'trigger', 'oxygen saturation'],\n",
       " 'SamplingFrequency': 100,\n",
       " 'StartTime': 0}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print all the metadata associated with this file\n",
    "bf.get_metadata()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Columns': ['cardiac', 'respiratory', 'trigger', 'oxygen saturation'],\n",
       " 'SamplingFrequency': 100,\n",
       " 'StartTime': 0,\n",
       " 'acquisition': 'fullbrain',\n",
       " 'datatype': 'func',\n",
       " 'extension': 'tsv.gz',\n",
       " 'run': 1,\n",
       " 'session': '1',\n",
       " 'subject': '01',\n",
       " 'suffix': 'physio',\n",
       " 'task': 'rest'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# We can the union of both of the above in one shot like this\n",
    "bf.get_entities(metadata='all')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here are all the files associated with our target file in some way. Notice how we get back both the JSON sidecar for our target file, and the BOLD run that our target file contains physiological recordings for."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<BIDSJSONFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/task-rest_acq-fullbrain_run-1_physio.json'>,\n",
       " <BIDSImageFile filename='/Users/tal/Dropbox/Code/pybids/bids/tests/data/7t_trt/sub-01/ses-1/func/sub-01_ses-1_task-rest_acq-fullbrain_run-1_bold.nii.gz'>]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bf.get_associations()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "In cases where a file has a `.tsv.gz` or `.tsv` extension, it will automatically be created as a `BIDSDataFile`, and we can easily grab the contents as a pandas `DataFrame`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>onset</th>\n",
       "      <th>respiratory</th>\n",
       "      <th>cardiac</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>-0.757342</td>\n",
       "      <td>0.048933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.1</td>\n",
       "      <td>-0.796851</td>\n",
       "      <td>0.355185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.2</td>\n",
       "      <td>-0.833215</td>\n",
       "      <td>0.626669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.3</td>\n",
       "      <td>-0.866291</td>\n",
       "      <td>0.836810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.4</td>\n",
       "      <td>-0.895948</td>\n",
       "      <td>0.965038</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   onset  respiratory   cardiac\n",
       "0    0.0    -0.757342  0.048933\n",
       "1    0.1    -0.796851  0.355185\n",
       "2    0.2    -0.833215  0.626669\n",
       "3    0.3    -0.866291  0.836810\n",
       "4    0.4    -0.895948  0.965038"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Use a different test dataset--one that contains physio recording files\n",
    "data_path = os.path.join(get_test_data_path(), 'synthetic')\n",
    "layout2 = BIDSLayout(data_path)\n",
    "\n",
    "# Get the first physiological recording file\n",
    "recfile = layout2.get(suffix='physio')[0]\n",
    "\n",
    "# Get contents as a DataFrame and show the first few rows\n",
    "df = recfile.get_df()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While it would have been easy enough to read the contents of the file ourselves with pandas' `read_csv()` method, notice that in the above example, `get_df()` saved us the trouble of having to read the physiological recording file's metadata, pull out the column names and sampling rate, and add timing information.\n",
    "\n",
    "Mind you, if we don't *want* the timing information, we can ignore it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>respiratory</th>\n",
       "      <th>cardiac</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.757342</td>\n",
       "      <td>0.048933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.796851</td>\n",
       "      <td>0.355185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-0.833215</td>\n",
       "      <td>0.626669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.866291</td>\n",
       "      <td>0.836810</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-0.895948</td>\n",
       "      <td>0.965038</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   respiratory   cardiac\n",
       "0    -0.757342  0.048933\n",
       "1    -0.796851  0.355185\n",
       "2    -0.833215  0.626669\n",
       "3    -0.866291  0.836810\n",
       "4    -0.895948  0.965038"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "recfile.get_df(include_timing=False).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Other utilities\n",
    "\n",
    "### Filename parsing\n",
    "Say you have a filename, and you want to manually extract BIDS entities from it. The `parse_file_entities` method provides the facility:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:24.035216Z",
     "start_time": "2018-08-01T20:05:24.030080Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'subject': '01', 'run': 1, 'suffix': 'T2w', 'extension': 'nii.gz'}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "path = \"/a/fake/path/to/a/BIDS/file/sub-01_run-1_T2w.nii.gz\"\n",
    "layout.parse_file_entities(path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Path construction\n",
    "You may want to create valid BIDS filenames for files that are new or hypothetical that would sit within your BIDS project. This is useful when you know what entity values you need to write out to, but don't want to deal with looking up the precise BIDS file-naming syntax. In the example below, imagine we've created a new file containing stimulus presentation information, and we want to save it to a `.tsv.gz` file, per the BIDS naming conventions. All we need to do is define a dictionary with the name components, and `build_path` takes care of the rest (including injecting sub-directories!):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:24.049418Z",
     "start_time": "2018-08-01T20:05:24.044544Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'sub-01/func/sub-01_task-nback_run-2_bold.nii.gz'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "entities = {\n",
    "    'subject': '01',\n",
    "    'run': 2,\n",
    "    'task': 'nback',\n",
    "    'suffix': 'bold'\n",
    "}\n",
    "\n",
    "layout.build_path(entities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:24.064220Z",
     "start_time": "2018-08-01T20:05:24.058906Z"
    },
    "scrolled": false
   },
   "source": [
    "You can also use `build_path` in more sophisticated ways—for example, by defining your own set of matching templates that cover cases not supported by BIDS out of the box. For example, suppose you want to create a template for naming a new z-stat file. You could do something like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'sub-01_task-nback_run-2_z.nii.gz'"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define the pattern to build out of the components passed in the dictionary\n",
    "pattern = \"sub-{subject}[_ses-{session}]_task-{task}[_acq-{acquisition}][_rec-{reconstruction}][_run-{run}][_echo-{echo}]_{suffix<z>}.nii.gz\",\n",
    "\n",
    "entities = {\n",
    "    'subject': '01',\n",
    "    'run': 2,\n",
    "    'task': 'nback',\n",
    "    'suffix': 'z'\n",
    "}\n",
    "\n",
    "# Notice we pass the new pattern as the second argument\n",
    "layout.build_path(entities, pattern, validate=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in the above example, we set `validate=False` to ensure that the standard BIDS file validator doesn't run (because the pattern we defined isn't actually compliant with the BIDS specification)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading derivatives\n",
    "\n",
    "By default, `BIDSLayout` objects are initialized without scanning contained `derivatives/` directories. But you can easily ensure that all derivatives files are loaded and endowed with the extra structure specified in the [derivatives config file](https://github.com/bids-standard/pybids/blob/master/bids/layout/config/derivatives.json):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:24.228683Z",
     "start_time": "2018-08-01T20:05:24.066251Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "BIDS Layout: ...bids/bids/tests/data/synthetic | Subjects: 5 | Sessions: 10 | Runs: 10"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Define paths to root and derivatives folders\n",
    "root = os.path.join(get_test_data_path(), 'synthetic')\n",
    "layout2 = BIDSLayout(root, derivatives=True)\n",
    "layout2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `scope` argument to `get()` specifies which part of the project to look in. By default, valid values are `'bids'` (for the \"raw\" BIDS project that excludes derivatives) and `'derivatives'` (for all BIDS-derivatives files). You can also pass the names of individual derivatives pipelines (e.g., passing `'fmriprep'` would search only in a `/derivatives/fmriprep` folder). Either a string or a list of strings can be passed.\n",
    "\n",
    "The following call returns the filenames of all derivatives files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:24.245649Z",
     "start_time": "2018-08-01T20:05:24.230490Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/dataset_description.json',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-01/func/sub-01_ses-01_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-01/ses-02/func/sub-01_ses-02_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-01/func/sub-02_ses-01_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-02/ses-02/func/sub-02_ses-02_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-01/func/sub-03_ses-01_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-03/ses-02/func/sub-03_ses-02_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-01/func/sub-04_ses-01_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-04/ses-02/func/sub-04_ses-02_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-01/func/sub-05_ses-01_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-01_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-nback_run-02_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_desc-confounds_regressors.tsv.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-MNI152NLin2009cAsym_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-T1w_desc-brain_mask.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-T1w_desc-brain_mask.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-T1w_desc-preproc_bold.nii',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/sub-05/ses-02/func/sub-05_ses-02_task-rest_space-T1w_desc-preproc_bold.nii.gz',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/task-nback_bold.json',\n",
       " '/Users/tal/Dropbox/Code/pybids/bids/tests/data/synthetic/derivatives/fmriprep/task-rest_bold.json']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get all files in derivatives\n",
    "layout2.get(scope='derivatives', return_type='file')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exporting a `BIDSLayout` to a pandas `Dataframe`\n",
    "If you want a summary of all the files in your `BIDSLayout`, but don't want to have to iterate `BIDSFile` objects and extract their entities, you can get a nice bird's-eye view of your dataset using the `to_df()` method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:24.275066Z",
     "start_time": "2018-08-01T20:05:24.248179Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>entity</th>\n",
       "      <th>path</th>\n",
       "      <th>acquisition</th>\n",
       "      <th>datatype</th>\n",
       "      <th>extension</th>\n",
       "      <th>fmap</th>\n",
       "      <th>run</th>\n",
       "      <th>scans</th>\n",
       "      <th>session</th>\n",
       "      <th>subject</th>\n",
       "      <th>suffix</th>\n",
       "      <th>task</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>description</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tsv</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>participants</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>anat</td>\n",
       "      <td>nii.gz</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>T1map</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>anat</td>\n",
       "      <td>nii.gz</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>T1w</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>fmap</td>\n",
       "      <td>nii.gz</td>\n",
       "      <td>magnitude1</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>magnitude1</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "entity                                               path acquisition  \\\n",
       "0       /Users/tal/Dropbox/Code/pybids/bids/tests/data...         NaN   \n",
       "1       /Users/tal/Dropbox/Code/pybids/bids/tests/data...         NaN   \n",
       "2       /Users/tal/Dropbox/Code/pybids/bids/tests/data...         NaN   \n",
       "3       /Users/tal/Dropbox/Code/pybids/bids/tests/data...         NaN   \n",
       "4       /Users/tal/Dropbox/Code/pybids/bids/tests/data...         NaN   \n",
       "\n",
       "entity datatype extension        fmap  run scans session subject  \\\n",
       "0           NaN      json         NaN  NaN   NaN     NaN     NaN   \n",
       "1           NaN       tsv         NaN  NaN   NaN     NaN     NaN   \n",
       "2          anat    nii.gz         NaN  NaN   NaN       1      01   \n",
       "3          anat    nii.gz         NaN  NaN   NaN       1      01   \n",
       "4          fmap    nii.gz  magnitude1    1   NaN       1      01   \n",
       "\n",
       "entity        suffix task  \n",
       "0        description  NaN  \n",
       "1       participants  NaN  \n",
       "2              T1map  NaN  \n",
       "3                T1w  NaN  \n",
       "4         magnitude1  NaN  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert the layout to a pandas dataframe\n",
    "df = layout.to_df()\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also include metadata in the result if we like (which may blow up our `DataFrame` if we have a large dataset). Note that in this case, most of our cells will have missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>entity</th>\n",
       "      <th>path</th>\n",
       "      <th>CogAtlasID</th>\n",
       "      <th>Columns</th>\n",
       "      <th>EchoTime</th>\n",
       "      <th>EchoTime1</th>\n",
       "      <th>EchoTime2</th>\n",
       "      <th>EffectiveEchoSpacing</th>\n",
       "      <th>IntendedFor</th>\n",
       "      <th>PhaseEncodingDirection</th>\n",
       "      <th>RepetitionTime</th>\n",
       "      <th>...</th>\n",
       "      <th>acquisition</th>\n",
       "      <th>datatype</th>\n",
       "      <th>extension</th>\n",
       "      <th>fmap</th>\n",
       "      <th>run</th>\n",
       "      <th>scans</th>\n",
       "      <th>session</th>\n",
       "      <th>subject</th>\n",
       "      <th>suffix</th>\n",
       "      <th>task</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>json</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>description</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>tsv</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>participants</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>anat</td>\n",
       "      <td>nii.gz</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>T1map</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>anat</td>\n",
       "      <td>nii.gz</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>T1w</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>/Users/tal/Dropbox/Code/pybids/bids/tests/data...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>fmap</td>\n",
       "      <td>nii.gz</td>\n",
       "      <td>magnitude1</td>\n",
       "      <td>1</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>magnitude1</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "entity                                               path CogAtlasID Columns  \\\n",
       "0       /Users/tal/Dropbox/Code/pybids/bids/tests/data...        NaN     NaN   \n",
       "1       /Users/tal/Dropbox/Code/pybids/bids/tests/data...        NaN     NaN   \n",
       "2       /Users/tal/Dropbox/Code/pybids/bids/tests/data...        NaN     NaN   \n",
       "3       /Users/tal/Dropbox/Code/pybids/bids/tests/data...        NaN     NaN   \n",
       "4       /Users/tal/Dropbox/Code/pybids/bids/tests/data...        NaN     NaN   \n",
       "\n",
       "entity EchoTime EchoTime1 EchoTime2 EffectiveEchoSpacing IntendedFor  \\\n",
       "0           NaN       NaN       NaN                  NaN         NaN   \n",
       "1           NaN       NaN       NaN                  NaN         NaN   \n",
       "2           NaN       NaN       NaN                  NaN         NaN   \n",
       "3           NaN       NaN       NaN                  NaN         NaN   \n",
       "4           NaN       NaN       NaN                  NaN         NaN   \n",
       "\n",
       "entity PhaseEncodingDirection RepetitionTime  ... acquisition datatype  \\\n",
       "0                         NaN            NaN  ...         NaN      NaN   \n",
       "1                         NaN            NaN  ...         NaN      NaN   \n",
       "2                         NaN            NaN  ...         NaN     anat   \n",
       "3                         NaN            NaN  ...         NaN     anat   \n",
       "4                         NaN            NaN  ...         NaN     fmap   \n",
       "\n",
       "entity extension        fmap  run scans session subject        suffix task  \n",
       "0           json         NaN  NaN   NaN     NaN     NaN   description  NaN  \n",
       "1            tsv         NaN  NaN   NaN     NaN     NaN  participants  NaN  \n",
       "2         nii.gz         NaN  NaN   NaN       1      01         T1map  NaN  \n",
       "3         nii.gz         NaN  NaN   NaN       1      01           T1w  NaN  \n",
       "4         nii.gz  magnitude1    1   NaN       1      01    magnitude1  NaN  \n",
       "\n",
       "[5 rows x 25 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "layout.to_df(metadata=True).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieving BIDS variables \n",
    "BIDS variables are stored in .tsv files at the run, session, subject, or dataset level. You can retrieve these variables with `layout.get_collections()`. The resulting objects can be converted to dataframes and merged with the layout to associate the variables with corresponding scans.\n",
    "\n",
    "In the following example, we request all subject-level variable data available anywhere in the BIDS project, and merge the results into a single `DataFrame` (by default, we'll get back a single `BIDSVariableCollection` object for each subject). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:26.693372Z",
     "start_time": "2018-08-01T20:05:24.276799Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>suffix</th>\n",
       "      <th>session</th>\n",
       "      <th>subject</th>\n",
       "      <th>CCPT_FN_count</th>\n",
       "      <th>CCPT_FP_count</th>\n",
       "      <th>CCPT_avg_FN_RT</th>\n",
       "      <th>CCPT_avg_FP_RT</th>\n",
       "      <th>CCPT_avg_succ_RT</th>\n",
       "      <th>CCPT_succ_count</th>\n",
       "      <th>caffeine_daily</th>\n",
       "      <th>...</th>\n",
       "      <th>relative_water_intake</th>\n",
       "      <th>specific_vague</th>\n",
       "      <th>subject_id</th>\n",
       "      <th>surroundings</th>\n",
       "      <th>systolic_blood_pressure_left</th>\n",
       "      <th>systolic_blood_pressure_right</th>\n",
       "      <th>thirst</th>\n",
       "      <th>vigilance</th>\n",
       "      <th>vigilance_nyc-q</th>\n",
       "      <th>words</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>sessions</td>\n",
       "      <td>1</td>\n",
       "      <td>01</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>507.0</td>\n",
       "      <td>500.770833</td>\n",
       "      <td>96.0</td>\n",
       "      <td>0.5</td>\n",
       "      <td>...</td>\n",
       "      <td>7.0</td>\n",
       "      <td>95.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>108.0</td>\n",
       "      <td>109.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>9.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>sessions</td>\n",
       "      <td>1</td>\n",
       "      <td>02</td>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>297.6</td>\n",
       "      <td>351.729167</td>\n",
       "      <td>96.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>99.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>sessions</td>\n",
       "      <td>1</td>\n",
       "      <td>03</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>441.0</td>\n",
       "      <td>426.718750</td>\n",
       "      <td>96.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>3.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>122.0</td>\n",
       "      <td>128.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>8.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>sessions</td>\n",
       "      <td>1</td>\n",
       "      <td>04</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>443.0</td>\n",
       "      <td>417.906250</td>\n",
       "      <td>96.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>...</td>\n",
       "      <td>5.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>130.0</td>\n",
       "      <td>110.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>85.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>sessions</td>\n",
       "      <td>1</td>\n",
       "      <td>05</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>355.5</td>\n",
       "      <td>372.114583</td>\n",
       "      <td>96.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>5.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>117.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 97 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     suffix session subject  CCPT_FN_count  CCPT_FP_count  CCPT_avg_FN_RT  \\\n",
       "0  sessions       1      01            0.0            1.0             NaN   \n",
       "1  sessions       1      02            0.0            5.0             NaN   \n",
       "2  sessions       1      03            0.0            1.0             NaN   \n",
       "3  sessions       1      04            0.0            1.0             NaN   \n",
       "4  sessions       1      05            0.0            2.0             NaN   \n",
       "\n",
       "   CCPT_avg_FP_RT  CCPT_avg_succ_RT  CCPT_succ_count  caffeine_daily  ...  \\\n",
       "0           507.0        500.770833             96.0             0.5  ...   \n",
       "1           297.6        351.729167             96.0             0.0  ...   \n",
       "2           441.0        426.718750             96.0             1.0  ...   \n",
       "3           443.0        417.906250             96.0             0.1  ...   \n",
       "4           355.5        372.114583             96.0             0.0  ...   \n",
       "\n",
       "   relative_water_intake  specific_vague  subject_id  surroundings  \\\n",
       "0                    7.0            95.0         1.0           0.0   \n",
       "1                    1.0           100.0         2.0          70.0   \n",
       "2                    3.0           100.0         3.0          10.0   \n",
       "3                    5.0            80.0         4.0           0.0   \n",
       "4                    5.0            75.0         5.0          80.0   \n",
       "\n",
       "   systolic_blood_pressure_left  systolic_blood_pressure_right  thirst  \\\n",
       "0                         108.0                          109.0     9.0   \n",
       "1                          99.0                          100.0     2.0   \n",
       "2                         122.0                          128.0     3.0   \n",
       "3                         130.0                          110.0     6.0   \n",
       "4                         105.0                          117.0     7.0   \n",
       "\n",
       "   vigilance  vigilance_nyc-q  words  \n",
       "0        9.0            100.0  100.0  \n",
       "1        7.0            100.0  100.0  \n",
       "2        8.0            100.0    0.0  \n",
       "3        5.0            100.0   85.0  \n",
       "4        7.0             60.0   30.0  \n",
       "\n",
       "[5 rows x 97 columns]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Get subject variables as a dataframe and merge them back in with the layout\n",
    "subj_df = layout.get_collections(level='subject', merge=True).to_df()\n",
    "subj_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## BIDSValidator\n",
    "\n",
    "`pybids` implicitly imports a `BIDSValidator` class from the separate [`bids-validator`](https://github.com/bids-standard/bids-validator) package. You can use the `BIDSValidator` to determine whether a filepath is a valid BIDS filepath, as well as answering questions about what kind of data it represents. Note, however, that this implementation of the BIDS validator is *not* necessarily up-to-date with the JavaScript version available online. Moreover, the Python validator only tests individual files, and is currently unable to validate entire BIDS datasets. For that, you should use the [online BIDS validator](https://bids-standard.github.io/bids-validator/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:27.763448Z",
     "start_time": "2018-08-01T20:05:27.749652Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from bids import BIDSValidator\n",
    "\n",
    "# Note that when using the bids validator, the filepath MUST be relative to the top level bids directory\n",
    "validator = BIDSValidator()\n",
    "validator.is_bids('/sub-02/ses-01/anat/sub-02_ses-01_T2w.nii.gz')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:27.773052Z",
     "start_time": "2018-08-01T20:05:27.765263Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Can decide if a filepath represents a file part of the specification\n",
    "validator.is_file('/sub-02/ses-01/anat/sub-02_ses-01_T2w.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Can check if a file is at the top level of the dataset\n",
    "validator.is_top_level('/dataset_description.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# or subject (or session) level\n",
    "validator.is_subject_level('/dataset_description.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "validator.is_session_level('/sub-02/ses-01/sub-02_ses-01_scans.json')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-08-01T20:05:27.780022Z",
     "start_time": "2018-08-01T20:05:27.775281Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Can decide if a filepath represents phenotypic data\n",
    "validator.is_phenotypic('/sub-02/ses-01/anat/sub-02_ses-01_T2w.nii.gz')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}