skips_delays.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "atomic-hamburg",
   "metadata": {
    "pycharm": {
     "name": "#%% md\n"
    }
   },
   "source": [
    "# Skips and delays analysis\n",
    "\n",
    "During the study, participants could skip any film clip.\n",
    "To see if a skip occurred, the signal recorded during stimulus was compared with the original film clip duration.\n",
    "\n",
    "Delays in the stimuli phase were caused by connection errors with the application server.\n",
    "Delays in the washout phase were caused by the reaction times of subjects.\n",
    "\n",
    "All data are available in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "dependent-yesterday",
   "metadata": {
    "pycharm": {
     "is_executing": true
    }
   },
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import configparser\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from tqdm.notebook import tqdm\n",
    "from pprint import pprint\n",
    "from datetime import datetime\n",
    "from helpers import STIMULATION_TIMES"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "based-aging",
   "metadata": {},
   "outputs": [],
   "source": [
    "config = configparser.ConfigParser()\n",
    "config.read(\"config.ini\")\n",
    "\n",
    "if not os.path.exists(config['DataDirectories']['unzipped_dataset']):\n",
    "    raise Exception(\"Please set path for unzipped dataset in config.ini\")\n",
    "\n",
    "DATA_DIR = config['DataDirectories']['unzipped_dataset']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "digital-advice",
   "metadata": {},
   "outputs": [],
   "source": [
    "# generate paths\n",
    "washout_paths, stimuli_paths = [], []\n",
    "\n",
    "for part in os.listdir(DATA_DIR):\n",
    "    for f_name in os.listdir(DATA_DIR + '/' + part):\n",
    "        if 'EMPATICA' in f_name:\n",
    "            if 'WASHOUT' in f_name:\n",
    "                # files recorded during washouts\n",
    "                washout_paths.append((part, f_name.split('_')[1], DATA_DIR + '/' + part + '/' + f_name))\n",
    "            elif 'STIMULUS' in f_name:\n",
    "                # files recorded during stimuli\n",
    "                stimuli_paths.append((part, f_name.split('_')[1], DATA_DIR + '/' + part + '/' + f_name))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "planned-avatar",
   "metadata": {},
   "source": [
    "## Stimuli"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "diagnostic-difficulty",
   "metadata": {},
   "source": [
    "Skip (shorter duration) means that the participant skipped some part of the film clip. The amount of skipped time is stored in the `skips` dictionary. Tolerance time for skips was set to 1s.\n",
    "\n",
    "Delay (longer duration) means that the participant watched the whole film clip, but the questionnaire did not load instantly. The amount of delay time is stored in the `delays` dictionary. The tolerance time for delays was set to 5s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "impossible-patent",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "758dd2a70e8d41b9817bde1ec37e70ad",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/464 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "skip_delta = pd.Timedelta('00:00:01')\n",
    "delay_delta = pd.Timedelta('00:00:00') - pd.Timedelta('00:00:05')\n",
    "skips, delays = dict(), dict()\n",
    "\n",
    "for part, emotion, path in tqdm(stimuli_paths):\n",
    "    if emotion == 'BASELINE':\n",
    "        continue\n",
    "    with open(path, 'r') as f:\n",
    "        bvp_times = [datetime.strptime(p[0], '%Y-%m-%dT%H:%M:%S:%f') for p in json.load(f)['BVP']]\n",
    "    \n",
    "    diff = STIMULATION_TIMES[emotion] - (bvp_times[-1] - bvp_times[0])\n",
    "    if diff > skip_delta:\n",
    "        # if skip occured add to skips\n",
    "        skips.setdefault(part, [])\n",
    "        skips[part].append((emotion, diff))\n",
    "    elif diff < delay_delta:\n",
    "        # if delay occured add to delays\n",
    "        delays.setdefault(part, [])\n",
    "        delays[part].append((emotion, pd.Timedelta('00:00:00') - diff))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "satisfactory-eclipse",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'37': [('FEAR', Timedelta('0 days 00:01:47.015031'))],\n",
      " '58': [('AWE', Timedelta('0 days 00:01:06.013337')),\n",
      "        ('DISGUST', Timedelta('0 days 00:00:04.012696')),\n",
      "        ('ENTHUSIASM', Timedelta('0 days 00:01:36.014573')),\n",
      "        ('LIKING', Timedelta('0 days 00:00:28.027452'))],\n",
      " '62': [('FEAR', Timedelta('0 days 00:01:22.013886'))]}\n"
     ]
    }
   ],
   "source": [
    "pprint(skips)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "consecutive-there",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'44': [('ENTHUSIASM', Timedelta('0 days 00:08:28.981870'))]}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "delays"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "infectious-architecture",
   "metadata": {},
   "source": [
    "## Washouts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "featured-custody",
   "metadata": {},
   "source": [
    "Skip (shorter duration) means that only part of the washout was presented. The amount of skipped time is stored in the `skips` dictionary. Tolerance time for skips was set to 1s.\n",
    "\n",
    "Delay (longer duration) means that the washout phase lasted longer than the washout film clip. In general, such delays are caused by the participant's reaction time, as subjects had to confirm they wanted to watch the next stimulus film clip after the washout ended. The amount of delay time is stored in the `delays` dictionary. The tolerance time for delays was set to 5s. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "labeled-turkey",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6b0bdc64691a4282b1c6372d81fb6e87",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/422 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "skip_delta = pd.Timedelta('00:00:01')\n",
    "delay_delta = pd.Timedelta('00:00:00') - pd.Timedelta('00:00:05')\n",
    "skips, delays = dict(), dict()\n",
    "\n",
    "for part, emotion, path in tqdm(washout_paths):\n",
    "    with open(path, 'r') as f:\n",
    "        bvp_times = [datetime.strptime(p[0], '%Y-%m-%dT%H:%M:%S:%f') for p in json.load(f)['BVP']]\n",
    "    \n",
    "    diff = STIMULATION_TIMES['WASHOUT'] - (bvp_times[-1] - bvp_times[0])\n",
    "    if diff > skip_delta:\n",
    "        # if skip occured add to skips\n",
    "        skips.setdefault(part, [])\n",
    "        skips[part].append((emotion, diff))\n",
    "    elif diff < delay_delta:\n",
    "        # if delay occured add to delays\n",
    "        delays.setdefault(part, [])\n",
    "        delays[part].append((emotion, pd.Timedelta('00:00:00') - diff))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "foreign-portal",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'48': [('FEAR', Timedelta('0 days 00:00:32.527246'))]}\n"
     ]
    }
   ],
   "source": [
    "# shorter washout for the participant 48 occured due to a application malfunction\n",
    "# after that, the rest of data from empatica is lost for this participant\n",
    "pprint(skips)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "separated-stephen",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All washouts: 422\n",
      "Wahouts longer than delay_time: 421\n"
     ]
    }
   ],
   "source": [
    "# all washouts other than fear (cell above) were longer than delay_time \n",
    "print('All washouts:', len(washout_paths))\n",
    "print('Wahouts longer than delay_time:', len([t[1] for times_list in delays.values() for t in times_list]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "geological-pavilion",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timedelta('0 days 00:00:08.746948824')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# avg delay\n",
    "np.mean([t[1] for times_list in delays.values() for t in times_list])"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}