Day1/Lab1_preprocessing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "26142471-4b25-4ac9-b791-bf1c2f78d870",
   "metadata": {},
   "source": [
    "<h1><center>NIH MEG Workshop 2023 Preprocessing </center></h1> "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05a83854-39ab-4df3-b437-174548adf11f",
   "metadata": {},
   "source": [
    "## General Import and settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b7b76e2-d5dd-4407-8014-689a17176002",
   "metadata": {},
   "outputs": [],
   "source": [
    "import mne\n",
    "import mne_bids\n",
    "import os, os.path as op\n",
    "import numpy as np\n",
    "import warnings\n",
    "import copy\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73ebe4e4-9c02-4885-aa64-35df76cebffd",
   "metadata": {},
   "source": [
    "# Setup Paths"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "309ea2f0-7659-4b16-a3b5-4e0fce0fbf21",
   "metadata": {},
   "source": [
    "## Set some generic BIDS information\n",
    "BIDS is a \"new\" standard for formatting neuroimaging/neurophysiology data <br>\n",
    "The bids_root is the top level, derivatives holds the projects, project level has the outputs for each subject"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "852aa6be-93c9-47ee-a0bd-ce2f7cb72dc7",
   "metadata": {},
   "source": [
    "| Folder | Data Type | \n",
    "| --- | --- | \n",
    "| bids_root | raw data |\n",
    "| deriv_root | folders for all possible derivatives | \n",
    "| project_root | [Derivatives]: output folder for this processing | \n",
    "| subjects_dir | [Derivatives]: freesurfer anatomical information | "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6be5157-fea4-4f69-88cc-11d16ccd9c56",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Raw data\n",
    "bids_root = op.join('/data/', os.environ['USER'], 'meg_workshop_data')  \n",
    "#Processed Data Folder: contains (freesurfer / Day1 / Day2) \n",
    "deriv_root = op.join(bids_root, 'derivatives')\n",
    "if not op.exists(deriv_root): os.mkdir(deriv_root)\n",
    "#This course data - bids derivatives outputs for Day1 \n",
    "project_root = op.join(deriv_root, 'Day1') \n",
    "#Freesurfer brain surface reconstruction\n",
    "fs_subjects_dir = op.join(deriv_root, 'freesurfer','subjects')\n",
    "subjects_dir = fs_subjects_dir"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce88eba3-891d-4cc2-b827-386ecddbecbb",
   "metadata": {},
   "source": [
    "### Find data for a subject"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f55f7bec-eefc-4acf-9658-13f7d75efe9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "subject = 'ON02811'\n",
    "bids_path = mne_bids.BIDSPath(root=bids_root, subject = subject, session='01', task='airpuff', run='01', datatype='meg')\n",
    "project_path = bids_path.copy().update(root=project_root, check=False)\n",
    "if not op.exists(project_path.root): os.mkdir(project_path.root)\n",
    "\n",
    "t1_bids_path = bids_path.copy().update(datatype='anat', session='01', task=None, run=None, suffix='T1w', acquisition='MPRAGE', extension='.nii.gz')\n",
    "print(f'Here is the meg data file: {bids_path.fpath}')\n",
    "print(f'Here is the t1 MRI file: {t1_bids_path.fpath}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "674e3b95-1e7a-4d33-ab72-709c02151971",
   "metadata": {},
   "source": [
    "## Load the MEG file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfc33de4-5e7d-4ae3-8766-06c91ca4bcfa",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Using the BIDS reader -- same as below\n",
    "raw = mne_bids.read_raw_bids(bids_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d563a75-f70e-4e9e-a50f-811cb321e3d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the data with the traditional reader -- same as above\n",
    "# raw = mne.io.read_raw_ctf(bids_path.fpath, clean_names=True, system_clock='ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40609884-43b3-463b-8fba-b5fbd6086d7a",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.info        #Note that this data has been anonymized (dates and names are scrubbed)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "561e27b1-ddf0-47ae-9bfb-7cbe744ce12c",
   "metadata": {},
   "source": [
    "| Channel Types | Function | |\n",
    "| --- | --- | --- |\n",
    "| MEG  |  MEG   | Collect Brain Signal |\n",
    "| Ref   | Reference Channels | Used for third Order Gradient to reject artifact | \n",
    "| UPPT | Digital Trigger  - Parrallel Port | designates trigger value - Up to 256| \n",
    "| UADC | Analog Triggers |  designates timing - eg Projector Trigger |  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa9a9a8b-0225-4cfd-93fc-888cac7022a6",
   "metadata": {},
   "source": [
    "#### 3rd Order Gradiometers applied - In our case 3rd order compensation is set by default during acquisition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61efb8c7-cc51-4771-97b6-d657de9bb5d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.compensation_grade"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4c54899f-e93b-47f2-9ba7-c387074731bb",
   "metadata": {},
   "source": [
    "## Plot **without** 3rd Order Gradient Compensation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff6263f4-68dc-4b7d-b298-9f7cf8ba8d92",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "_=raw.copy().apply_gradient_compensation(1).plot(n_channels=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c430e51c-2225-4bb1-8f4a-982bfa08af60",
   "metadata": {},
   "source": [
    "## Plot **with** 3rd Order Gradient"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "413feb9c-b04f-4fdb-8d20-74364dc35e90",
   "metadata": {},
   "outputs": [],
   "source": [
    "print('3rd Order Gradient applied')\n",
    "_=raw.plot(n_channels=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85e6944d-c3f2-4169-ae86-513d48826648",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.load_data()\n",
    "print(f'The data consists of a matrix: {raw._data.shape[0]} channels by {raw._data.shape[1]} samples')\n",
    "print(f\"This is samples / sampling frequency: {raw._data.shape[1]/raw.info['sfreq']} seconds\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e9604eb6-15df-442d-b0de-38e21d7a5645",
   "metadata": {},
   "source": [
    "## Channel information"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2d697c4-63c6-4496-bd0d-9604530df8c2",
   "metadata": {},
   "source": [
    "![SensorLayout](images/sens_layout.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "26b64428-e7eb-4edb-af19-a3a36ba16f83",
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Some examples of channel names')\n",
    "raw.ch_names[50:110:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87be1555-2bb2-4a9f-a070-b23533253575",
   "metadata": {},
   "source": [
    "### Selecting channels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b7f9b1c-18e7-4119-a4a3-e16c82e2a07c",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# Also look at raw.pick_channels and raw.pick\n",
    "# raw.pick_types?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6d673e00-d568-4b16-9e4e-3f790196e141",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.info"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ae7511a-64cb-4172-bc40-9d7c0d74e8a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.pick_types(meg=True, eeg=False, misc=True)._data.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "212261c8-8380-48ba-8ab7-815db7e3923a",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'There are {raw.copy().pick_types(meg=True, ref_meg=False)._data.shape[0]} meg channels and {raw.copy().pick_types(meg=False, ref_meg=True)._data.shape[0]} ref channels')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27cb064f-f200-45f6-b079-576235699da3",
   "metadata": {},
   "source": [
    "## Processing Triggers"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "27185195-132e-46df-b8ea-94bb7fb64d63",
   "metadata": {},
   "source": [
    "### The data is collected continuously while the subject is presented with stimuli (eg auditory, visual, somatosensory).  The triggers are then used to pull information from the continous dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5bf0c6b-aa69-4c4c-967f-be75f098881a",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"UADC001 index: {raw.ch_names.index('UADC001')}\")\n",
    "print(f\"UPPT001 index: {raw.ch_names.index('UPPT001')}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8bfe5eec-c0da-4341-9ad4-205080dc358b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#General plotting using python plotting\n",
    "%matplotlib inline\n",
    "samp_start = 1000\n",
    "samp_end = 10000\n",
    "fig, ax = plt.subplots(2,2)\n",
    "#Note the indexing from previous cell (301 is UADC, 302 is UPPT)\n",
    "ax[0,0].plot(raw._data[301, samp_start:samp_end])   \n",
    "ax[0,0].set_title('UADC001')\n",
    "ax[0,1].plot(raw._data[302, samp_start:samp_end])\n",
    "ax[0,1].set_title('UPPT001 - multiple vals')\n",
    "ax[1,0].plot(raw._data[301, samp_start:int(samp_end/7)])\n",
    "ax[1,0].set_title('UADC001 (analog) - Zoomed')\n",
    "ax[1,1].plot(raw._data[302, samp_start:int(samp_end/7)])\n",
    "ax[1,1].set_title('UPPT001 (digital) - Zoomed')\n",
    "fig.suptitle('Notice the difference between the Trigger Types')\n",
    "fig.tight_layout()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b462fb93-37bc-4fad-910d-d8f3ec5d344c",
   "metadata": {},
   "source": [
    "### Somatosensory task - Airpuff Stimulus (Stim) vs. Missing Stim \n",
    "### UPPT001 codes stimuli; UADC001 communicates to mechanical airpuff delivery system"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ddfedea-f46e-4ac7-9efa-92a0122c4f0b",
   "metadata": {},
   "source": [
    "### Sensor level preprocessing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc297a68-df25-4603-bc14-22ad15351771",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw = raw.load_data() "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a037937-dce9-4190-9252-a687015045b2",
   "metadata": {},
   "source": [
    "# Creating Events and Epochs from the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b5b5112-0462-4ba5-a72e-56ce629fe249",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.annotations  #These have been read from the MarkerFile.mrk upon reading "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c15b33db-5373-4e34-adaa-59fa8ea1b83d",
   "metadata": {},
   "outputs": [],
   "source": [
    "events, event_ids = mne.events_from_annotations(raw) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5aaeba77-7ae6-4fb0-9fd3-d9583e863aa0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dictionary that maps event name to event code\n",
    "event_ids"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "021a1307-5952-4f51-a4e1-313a840e1798",
   "metadata": {},
   "outputs": [],
   "source": [
    "events    # [Sample , duration, stim_code]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c340f0b5-dc1b-4e4f-8ca0-3ca62a42b90b",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "_ = mne.viz.plot_events(events, sfreq=raw.info['sfreq'], event_id=event_ids)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d00122a3-ad64-4871-a2fa-93c58be71c22",
   "metadata": {},
   "source": [
    "## Epochs: Time windows of data around each event "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "506a593b-4ab1-4861-a9aa-52668a04bed5",
   "metadata": {},
   "outputs": [],
   "source": [
    "#At time 0ms the stimuli is delivered\n",
    "tmin=-0.1  #100ms before event\n",
    "tmax=0.2   #300ms after event\n",
    "\n",
    "#Calculate all epochs with no rejection\n",
    "epochs = mne.Epochs(raw, events, tmin=tmin, tmax=tmax, reject_tmax=0,\n",
    "                preload=True, baseline=(tmin, 0))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "28cc02d0-3eff-4400-a147-ce1a6a3f674a",
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Number of Epochs X Number of Channels X Number of samples')\n",
    "print(epochs._data.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d0d35523-62ed-457e-98b0-993abac29519",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib qt\n",
    "_=epochs.plot(n_channels=10, events=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc37d93a-27c3-4a5c-a32d-474f0efe2b42",
   "metadata": {},
   "source": [
    "### Rejecting bad epochs using amplitude thresholding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bdf33726-fe9b-4d9c-9a91-618877e37af8",
   "metadata": {},
   "outputs": [],
   "source": [
    "reject_dict = {'mag':2500e-15}\n",
    "cleaned=epochs.copy().drop_bad(reject=reject_dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acf19b07-1fd4-41a2-bfe9-9028a8e6876e",
   "metadata": {},
   "source": [
    "### Epochs data size: Epochs X Channels X TimeSteps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "916d9fca-1801-4727-93bf-680ce357df57",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'Original Data Size:{epochs._data.shape}')\n",
    "print(f'Cleaned  Data Size:{cleaned._data.shape}')\n",
    "print('The matrix is now 6 epochs less')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a745cd2f-9e0f-4b29-b2af-e7de60c1d7e8",
   "metadata": {},
   "source": [
    "### Evoked data averages over epochs: Channels X TimeSteps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a4fcf840-ccb2-4448-9c9e-093fa1b402ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "evoked = epochs['2'].average()\n",
    "evoked._data.shape  #Also removes 4 response/stim/clock channels\n",
    "print('The evoked data averages all of the epochs')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2682c7a3-8ed0-4075-b5d1-662ec2f8c414",
   "metadata": {},
   "source": [
    "### Butterfly plot of evoked brain activity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "763ae06c-5520-4a5c-abb6-4041d83cd48f",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "_=evoked.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca953e11-e27b-43c6-b46e-54f5623bd26d",
   "metadata": {},
   "source": [
    "### Plot evoked dataset over time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71da1d69-00cd-456d-a01f-d1e8157d721c",
   "metadata": {},
   "outputs": [],
   "source": [
    "_=evoked.plot_topomap(times=np.arange(0,.1, 0.01))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07b787ef-21ec-4d12-9381-7f7766a25102",
   "metadata": {},
   "source": [
    "### Effect of number of samples on data average"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3070b59c-aad5-4298-a1c7-6e999a8a5a64",
   "metadata": {},
   "outputs": [],
   "source": [
    "stim_epochs=epochs['2']\n",
    "stim_epochs.pick_types(meg=True, ref_meg=False)\n",
    "bg_noise={}\n",
    "\n",
    "%matplotlib inline\n",
    "fig,axes = plt.subplots(3,2)\n",
    "idx=0\n",
    "reduce = [1,2,4,8,16,32]\n",
    "for row in axes: \n",
    "    for ax in row: \n",
    "        evoked = stim_epochs[0::reduce[idx]].average(method='mean')\n",
    "        ax.plot(evoked.times, evoked._data.T)\n",
    "        ax.set_ylim(-200e-15, 200e-15)\n",
    "        ax.set_title(f'# of Epochs in Average: {evoked.nave}')\n",
    "        bg_noise[evoked.nave]=(evoked._data[:,np.where(evoked.times<0)[0]]**2).sum()\n",
    "        idx+=1\n",
    "fig.suptitle('Effects of epoch number on average')\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1361e4fe-7bda-4cc4-8458-91c297bacdbd",
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.scatter(bg_noise.keys(), bg_noise.values()); plt.title('Background Prestim Noise versus Number of Epochs'); plt.xlabel('Epochs')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76b1e5ad-cd92-44ed-b2cb-2d7c79a4fe67",
   "metadata": {},
   "outputs": [],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "\n",
    "stim_epochs=epochs['2']\n",
    "\n",
    "%matplotlib qt\n",
    "fig, axes = plt.subplots(3,2)\n",
    "idx=0\n",
    "reduce = [1,2,4,8,16,32]\n",
    "for row in axes: \n",
    "    for ax in row:\n",
    "        evoked = stim_epochs[0::reduce[idx]].average(method='mean')\n",
    "        evoked.plot_topomap(times=[0.040], colorbar=False, axes=ax, scalings={'mag':1e15})\n",
    "        ax.set_title(f'40ms - Number Ave:{evoked.nave}')\n",
    "        idx+=1\n",
    "plt.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c06a082e-2220-45d8-bf66-3431ccb7f19d",
   "metadata": {},
   "source": [
    "## Adjusting event timing and creating new Events"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4cda4b3f-0fa7-46d4-83d2-51631233b3ab",
   "metadata": {},
   "outputs": [],
   "source": [
    "from nih2mne.utilities.markerfile_write import main as write_markerfile\n",
    "from nih2mne.utilities.trigger_utilities import threshold_detect, parse_marks, detect_digital, append_conditions\n",
    "#detect_digitial --- for digital triggers (UPPT001)\n",
    "#threshold_detect --- for analog triggers (UADC__)\n",
    "#parse_marks --- do some fancy fancy trigger stuff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e46c09a7-3208-4b86-9239-0a9dc0ccd6cb",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Changing task to the ODDball task\n",
    "bids_path.update(task='oddball')\n",
    "raw = mne_bids.read_raw_bids(bids_path)\n",
    "if op.exists(op.join(bids_path.fpath,'MarkerFile.mrk')): os.remove(op.join(bids_path.fpath,'MarkerFile.mrk'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d59cacd1-ea67-419f-9f01-519207539dea",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Get the onsets from the parrallel port\n",
    "ppt=detect_digital(bids_path.fpath)\n",
    "ppt #Output is a pandas tabular dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cf4bd33-545a-45eb-93f5-3dc707dc47af",
   "metadata": {},
   "outputs": [],
   "source": [
    "print([i for i in raw.ch_names if i[0:4]=='UADC'], 'These are the UADC channels in the data')\n",
    "print('UADC005 is the subject response code')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5eefb736-1a1d-482e-8aad-f3a6860f7cdd",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "uadcs = raw.copy().pick(['UPPT001','UADC005'])\n",
    "uadcs.plot(scalings=dict(misc=3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "898cad32-f6bc-4259-91db-729f9064df9d",
   "metadata": {},
   "source": [
    "## After reviewing the data (and also knowing the stim) - the PPT value 3 is the response target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1aac6e28-8eb1-41d0-b7f5-ae986e357b1e",
   "metadata": {},
   "outputs": [],
   "source": [
    "ppt.loc[ppt.condition=='1','condition'] = 'standard'\n",
    "ppt.loc[ppt.condition=='2','condition'] = 'distractor'\n",
    "ppt.loc[ppt.condition=='3','condition'] = 'target' "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "491a858f-6b08-45e2-9a80-7a8453990b79",
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Get the onsets of the subject_response device\n",
    "response=threshold_detect(str(bids_path.fpath), channel='UADC003', mark='response')\n",
    "response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2b5b8732-0fe1-4c00-a949-aeea09bea8d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "dframe= append_conditions([response, ppt])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7a76376-75dc-48b0-8585-daf9d52b1d7f",
   "metadata": {},
   "outputs": [],
   "source": [
    "dframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce8d0e34-bae1-4bbc-bdda-b37ed32a12ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "dframe=parse_marks(dframe, lead_condition='target', lag_condition='response', marker_on='lag', marker_name='CorrectResp')\n",
    "dframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e018bfe-a305-4a34-8eb5-1b8712996860",
   "metadata": {},
   "outputs": [],
   "source": [
    "dframe.condition.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec90f78e-2bb8-4a90-be73-9cc372fb73c8",
   "metadata": {},
   "source": [
    "### Annotations / events are not in the original file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d25b207d-27c2-409e-9be5-a22658c48286",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.annotations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e68ba901-0eef-434a-94b6-71947261fa34",
   "metadata": {},
   "outputs": [],
   "source": [
    "write_markerfile(dframe, bids_path.fpath)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a0bcc3a-f3e9-4f30-b6d4-df32d5151393",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw=mne_bids.read_raw_bids(bids_path, verbose=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66075e8c-ae9f-46eb-b79b-bf919c917c56",
   "metadata": {},
   "source": [
    "### The annotations have now been written into the CTF file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cefa03c0-32cb-44cf-ae92-da869a806816",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw.annotations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d83dfd0f-af28-48d3-b882-75eddd154135",
   "metadata": {},
   "source": [
    "#### Note that the vertical event designators are now present in the data when plotting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f2e9caa-118a-4faf-9c7f-6ff8c0e8c5fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "uadcs = raw.copy().pick(['UPPT001','UADC005'])\n",
    "uadcs.plot(scalings=dict(misc=3))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}