第9章 时序数据.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第9章 时序数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:27.967163Z",
     "start_time": "2020-06-29T09:52:27.255789Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、时序的创建\n",
    "###  1. 四类时间变量\n",
    "#### 现在理解可能关于③和④有些困惑，后面会作出一些说明"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "名称 | 描述 | 元素类型 | 创建方式  \n",
    ":-|:-|:-|:-\n",
    "① Date times（时间点/时刻） | 描述特定日期或时间点 | Timestamp | to_datetime或date_range\n",
    "② Time spans（时间段/时期） | 由时间点定义的一段时期 | Period | Period或period_range\n",
    "③ Date offsets（相对时间差） | 一段时间的相对大小（与夏/冬令时无关） | DateOffset | DateOffset\n",
    "④ Time deltas（绝对时间差） | 一段时间的绝对大小（与夏/冬令时有关） | Timedelta | to_timedelta或timedelta_range"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 时间点的创建"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （a）to_datetime方法\n",
    "#### Pandas在时间点建立的输入格式规定上给了很大的自由度，下面的语句都能正确建立同一时间点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:27.982120Z",
     "start_time": "2020-06-29T09:52:27.968165Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-01-01 00:00:00')"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.to_datetime('2020.1.1')\n",
    "pd.to_datetime('2020 1.1')\n",
    "pd.to_datetime('2020 1 1')\n",
    "pd.to_datetime('2020 1-1')\n",
    "pd.to_datetime('2020-1 1')\n",
    "pd.to_datetime('2020-1-1')\n",
    "pd.to_datetime('2020/1/1')\n",
    "pd.to_datetime('1.1.2020')\n",
    "pd.to_datetime('1.1 2020')\n",
    "pd.to_datetime('1 1 2020')\n",
    "pd.to_datetime('1 1-2020')\n",
    "pd.to_datetime('1-1 2020')\n",
    "pd.to_datetime('1-1-2020')\n",
    "pd.to_datetime('1/1/2020')\n",
    "pd.to_datetime('20200101')\n",
    "pd.to_datetime('2020.0101')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 下面的语句都会报错"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:27.998135Z",
     "start_time": "2020-06-29T09:52:27.983121Z"
    }
   },
   "outputs": [],
   "source": [
    "#pd.to_datetime('2020\\\\1\\\\1')\n",
    "#pd.to_datetime('2020`1`1')\n",
    "#pd.to_datetime('2020.1 1')\n",
    "#pd.to_datetime('1 1.2020')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 此时可利用format参数强制匹配"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.014149Z",
     "start_time": "2020-06-29T09:52:27.999136Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-01-01 00:00:00')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.to_datetime('2020\\\\1\\\\1',format='%Y\\\\%m\\\\%d')\n",
    "pd.to_datetime('2020`1`1',format='%Y`%m`%d')\n",
    "pd.to_datetime('2020.1 1',format='%Y.%m %d')\n",
    "pd.to_datetime('1 1.2020',format='%d %m.%Y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 同时，使用列表可以将其转为时间点索引"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.021437Z",
     "start_time": "2020-06-29T09:52:28.015150Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    0\n",
       "2020-01-02    1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(range(2),index=pd.to_datetime(['2020/1/1','2020/1/2']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.036429Z",
     "start_time": "2020-06-29T09:52:28.022077Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pandas.core.indexes.datetimes.DatetimeIndex"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(pd.to_datetime(['2020/1/1','2020/1/2']))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 对于DataFrame而言，如果列已经按照时间顺序排好，则利用to_datetime可自动转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.051617Z",
     "start_time": "2020-06-29T09:52:28.036429Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2020-01-01\n",
       "1   2020-01-02\n",
       "dtype: datetime64[ns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.DataFrame({'year': [2020, 2020],'month': [1, 1], 'day': [1, 2]})\n",
    "pd.to_datetime(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （b）时间精度与范围限制\n",
    "#### 事实上，Timestamp的精度远远不止day，可以最小到纳秒ns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.067631Z",
     "start_time": "2020-06-29T09:52:28.052618Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-01-01 00:00:00.123456789')"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.to_datetime('2020/1/1 00:00:00.123456789')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 同时，它带来范围的代价就是只有大约584年的时间点是可用的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.082645Z",
     "start_time": "2020-06-29T09:52:28.068632Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('1677-09-21 00:12:43.145225')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp.min"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.098659Z",
     "start_time": "2020-06-29T09:52:28.083646Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2262-04-11 23:47:16.854775807')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp.max"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （c）date_range方法\n",
    "#### 一般来说，start/end/periods（时间点个数）/freq（间隔方法）是该方法最重要的参数，给定了其中的3个，剩下的一个就会被确定"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.114303Z",
     "start_time": "2020-06-29T09:52:28.099660Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-01 00:00:00', '2020-01-05 12:00:00',\n",
       "               '2020-01-10 00:00:00'],\n",
       "              dtype='datetime64[ns]', freq=None)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(start='2020/1/1',end='2020/1/10',periods=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.121699Z",
     "start_time": "2020-06-29T09:52:28.115304Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03', '2020-01-04',\n",
       "               '2020-01-05', '2020-01-06', '2020-01-07', '2020-01-08',\n",
       "               '2020-01-09', '2020-01-10'],\n",
       "              dtype='datetime64[ns]', freq='D')"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(start='2020/1/1',end='2020/1/10',freq='D')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.137239Z",
     "start_time": "2020-06-29T09:52:28.122227Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03'], dtype='datetime64[ns]', freq='D')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(start='2020/1/1',periods=3,freq='D')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.153253Z",
     "start_time": "2020-06-29T09:52:28.137239Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-01', '2020-01-02', '2020-01-03'], dtype='datetime64[ns]', freq='D')"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(end='2020/1/3',periods=3,freq='D')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 其中freq参数有许多选项，下面将常用部分罗列如下，更多选项可看[这里](https://pandas.pydata.org/docs/user_guide/timeseries.html#offset-aliases)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "符号 | D/B | W | M/Q/Y | BM/BQ/BY | MS/QS/YS | BMS/BQS/BYS | H | T | S\n",
    ":-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:\n",
    "描述 | 日/工作日 | 周 | 月末 | 月/季/年末日 | 月/季/年末工作日 | 月/季/年初日 | 月/季/年初工作日 | 小时 | 分钟 |秒"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.169267Z",
     "start_time": "2020-06-29T09:52:28.153253Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-01 00:00:00', '2020-01-01 00:01:00',\n",
       "               '2020-01-01 00:02:00'],\n",
       "              dtype='datetime64[ns]', freq='T')"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(start='2020/1/1',periods=3,freq='T')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.187017Z",
     "start_time": "2020-06-29T09:52:28.169267Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-31', '2020-02-29', '2020-03-31'], dtype='datetime64[ns]', freq='M')"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(start='2020/1/1',periods=3,freq='M')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.203982Z",
     "start_time": "2020-06-29T09:52:28.187017Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-01', '2021-01-01', '2022-01-03'], dtype='datetime64[ns]', freq='BAS-JAN')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range(start='2020/1/1',periods=3,freq='BYS')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### bdate_range是一个类似与date_range的方法，特点在于可以在自带的工作日间隔设置上，再选择weekmask参数和holidays参数\n",
    "#### 它的freq中有一个特殊的'C'/'CBM'/'CBMS'选项，表示定制，需要联合weekmask参数和holidays参数使用\n",
    "#### 例如现在需要将工作日中的周一、周二、周五3天保留，并将部分holidays剔除"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:28.219995Z",
     "start_time": "2020-06-29T09:52:28.205983Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-03', '2020-01-06', '2020-01-13', '2020-01-14'], dtype='datetime64[ns]', freq='C')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weekmask = 'Mon Tue Fri'\n",
    "holidays = [pd.Timestamp('2020/1/%s'%i) for i in range(7,13)] #注意holidays\n",
    "pd.bdate_range(start='2020-1-1',end='2020-1-15',freq='C',weekmask=weekmask,holidays=holidays)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. DateOffset对象"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （a）DataOffset与Timedelta的区别\n",
    "#### Timedelta绝对时间差的特点指无论是冬令时还是夏令时，增减1day都只计算24小时\n",
    "#### DataOffset相对时间差指，无论一天是23\\24\\25小时，增减1day都与当天相同的时间保持一致\n",
    "#### 例如，英国当地时间 2020年03月29日，01:00:00 时钟向前调整 1 小时 变为 2020年03月29日，02:00:00，开始夏令时"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.586396Z",
     "start_time": "2020-06-29T09:52:28.225000Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-03-30 02:00:00+0300', tz='Europe/Helsinki')"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts = pd.Timestamp('2020-3-29 01:00:00', tz='Europe/Helsinki')\n",
    "ts + pd.Timedelta(days=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.602410Z",
     "start_time": "2020-06-29T09:52:29.587397Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-03-30 01:00:00+0300', tz='Europe/Helsinki')"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts + pd.DateOffset(days=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 这似乎有些令人头大，但只要把tz（time zone）去除就可以不用管它了，两者保持一致，除非要使用到时区变换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.618424Z",
     "start_time": "2020-06-29T09:52:29.603410Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-03-30 01:00:00')"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts = pd.Timestamp('2020-3-29 01:00:00')\n",
    "ts + pd.Timedelta(days=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.634439Z",
     "start_time": "2020-06-29T09:52:29.619426Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-03-30 01:00:00')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts + pd.DateOffset(days=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （b）增减一段时间\n",
    "#### DateOffset的可选参数包括years/months/weeks/days/hours/minutes/seconds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.650453Z",
     "start_time": "2020-06-29T09:52:29.635440Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-01-01 00:20:00')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp('2020-01-01') + pd.DateOffset(minutes=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2019-12-18 00:20:00')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp('2020-01-01') + pd.DateOffset(minutes=20) - pd.DateOffset(weeks=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （c）各类常用offset对象"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "freq | D/B | W | (B)M/(B)Q/(B)Y | (B)MS/(B)QS/(B)YS | H | T | S | C |\n",
    ":-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:|:-:\n",
    "offset | DateOffset/BDay | Week | (B)MonthEnd/(B)QuarterEnd/(B)YearEnd | (B)MonthBegin/(B)QuarterBegin/(B)YearBegin | Hour | Minute | Second | CDay(定制工作日)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.666467Z",
     "start_time": "2020-06-29T09:52:29.651454Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-01-15 00:00:00')"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp('2020-01-01') + pd.offsets.Week(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.682481Z",
     "start_time": "2020-06-29T09:52:29.667468Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Timestamp('2020-03-02 00:00:00')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Timestamp('2020-01-01') + pd.offsets.BQuarterBegin(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （d）序列的offset操作\n",
    "#### 利用apply函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.698495Z",
     "start_time": "2020-06-29T09:52:29.683482Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2023-01-02\n",
       "1   2024-01-01\n",
       "2   2025-01-01\n",
       "dtype: datetime64[ns]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(pd.offsets.BYearBegin(3).apply(i) for i in pd.date_range('20200101',periods=3,freq='Y'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 直接使用对象加减"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.714510Z",
     "start_time": "2020-06-29T09:52:29.699497Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2023-01-02', '2024-01-01', '2025-01-01'], dtype='datetime64[ns]', freq='A-DEC')"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range('20200101',periods=3,freq='Y') + pd.offsets.BYearBegin(3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 定制offset，可以指定weekmask和holidays参数（思考为什么三个都是一个值）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " 这里不懂"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.730524Z",
     "start_time": "2020-06-29T09:52:29.715511Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0   2020-01-15\n",
       "1   2020-01-15\n",
       "2   2020-01-15\n",
       "dtype: datetime64[ns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(pd.offsets.CDay(3,weekmask='Wed Fri',holidays='20200101').apply(i)\n",
    "                                  for i in pd.date_range('20200105',periods=3,freq='D'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.746539Z",
     "start_time": "2020-06-29T09:52:29.730524Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01-05 00:00:00\n",
      "2020-01-06 00:00:00\n",
      "2020-01-07 00:00:00\n"
     ]
    }
   ],
   "source": [
    "for i in pd.date_range('20200105',periods=3,freq='D'):\n",
    "    print(i)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "pandas.tseries.offsets.CDay  \n",
    "alias of pandas.tseries.offsets.CustomBusinessDay"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[pandas.tseries.offsets.CustomBusinessDay](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.tseries.offsets.CustomBusinessDay.html#pandas.tseries.offsets.CustomBusinessDay)  \n",
    "(n=1, normalize=False, weekmask='Mon Tue Wed Thu Fri', holidays=None, calendar=None, offset=datetime.timedelta(0))  \n",
    "DateOffset subclass representing possibly n custom business days, excluding holidays.  \n",
    "- n: int, default 1\n",
    "- normalize: bool, default False  \n",
    "Normalize start/end dates to midnight before generating date range.\n",
    "- weekmask: str, Default ‘Mon Tue Wed Thu Fri’  \n",
    "Weekmask of valid business days, passed to numpy.busdaycalendar.\n",
    "- holidays: list  \n",
    "List/array of dates to exclude from the set of valid business days, passed to numpy.busdaycalendar.\n",
    "- calendar: pd.HolidayCalendar or np.busdaycalendar  \n",
    "offsettimedelta, default timedelta(0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " 看不懂"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 二、时序的索引及属性\n",
    "### 1. 索引切片\n",
    "#### 这一部分几乎与第二章的规则完全一致"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.762553Z",
     "start_time": "2020-06-29T09:52:29.747540Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-05   -0.803194\n",
       "2020-01-12    1.143689\n",
       "2020-01-19   -0.612257\n",
       "2020-01-26   -0.582919\n",
       "2020-02-02    0.099017\n",
       "Freq: W-SUN, dtype: float64"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rng = pd.date_range('2020','2021', freq='W')\n",
    "ts = pd.Series(np.random.randn(len(rng)), index=rng)\n",
    "ts.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.778567Z",
     "start_time": "2020-06-29T09:52:29.763555Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "-0.5829194291579303"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts['2020-01-26']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 合法字符自动转换为时间点"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.794582Z",
     "start_time": "2020-06-29T09:52:29.779568Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-26   -0.582919\n",
       "2020-02-02    0.099017\n",
       "2020-02-09    0.201697\n",
       "2020-02-16    0.094219\n",
       "2020-02-23   -0.788263\n",
       "Freq: W-SUN, dtype: float64"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts['2020-01-26':'20200726'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 子集索引"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.810597Z",
     "start_time": "2020-06-29T09:52:29.795583Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-07-05    1.034421\n",
       "2020-07-12    0.047895\n",
       "2020-07-19   -0.368433\n",
       "2020-07-26   -1.091126\n",
       "Freq: W-SUN, dtype: float64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts['2020-7'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 支持混合形态索引"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.826610Z",
     "start_time": "2020-06-29T09:52:29.811597Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-05   -0.803194\n",
       "2020-01-12    1.143689\n",
       "2020-01-19   -0.612257\n",
       "2020-01-26   -0.582919\n",
       "2020-02-02    0.099017\n",
       "Freq: W-SUN, dtype: float64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts['2011-1':'20200726'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 时间点的属性"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 采用dt对象可以轻松获得关于时间的信息"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.842625Z",
     "start_time": "2020-06-29T09:52:29.826610Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    2\n",
       "2    3\n",
       "3    4\n",
       "4    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(ts.index).dt.week.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.858639Z",
     "start_time": "2020-06-29T09:52:29.842625Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     5\n",
       "1    12\n",
       "2    19\n",
       "3    26\n",
       "4     2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(ts.index).dt.day.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 利用strftime可重新修改时间格式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.874653Z",
     "start_time": "2020-06-29T09:52:29.859640Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    2020-间隔1-01-间隔2-05\n",
       "1    2020-间隔1-01-间隔2-12\n",
       "2    2020-间隔1-01-间隔2-19\n",
       "3    2020-间隔1-01-间隔2-26\n",
       "4    2020-间隔1-02-间隔2-02\n",
       "dtype: object"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.Series(ts.index).dt.strftime('%Y-间隔1-%m-间隔2-%d').head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 对于datetime对象可以直接通过属性获取信息"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.890668Z",
     "start_time": "2020-06-29T09:52:29.875654Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([ 1,  1,  1,  1,  2,  2,  2,  2,  3,  3,  3,  3,  3,  4,  4,  4,  4,\n",
       "             5,  5,  5,  5,  5,  6,  6,  6,  6,  7,  7,  7,  7,  8,  8,  8,  8,\n",
       "             8,  9,  9,  9,  9, 10, 10, 10, 10, 11, 11, 11, 11, 11, 12, 12, 12,\n",
       "            12],\n",
       "           dtype='int64')"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range('2020','2021', freq='W').month"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.906681Z",
     "start_time": "2020-06-29T09:52:29.891668Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,\n",
       "            6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6,\n",
       "            6, 6, 6, 6, 6, 6, 6, 6],\n",
       "           dtype='int64')"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.date_range('2020','2021', freq='W').weekday"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三、重采样"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 所谓重采样，就是指resample函数，它可以看做时序版本的groupby函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. resample对象的基本操作\n",
    "#### 采样频率一般设置为上面提到的offset字符"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.922696Z",
     "start_time": "2020-06-29T09:52:29.906681Z"
    }
   },
   "outputs": [],
   "source": [
    "df_r = pd.DataFrame(np.random.randn(1000, 3),index=pd.date_range('1/1/2020', freq='S', periods=1000),\n",
    "                  columns=['A', 'B', 'C'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.938710Z",
     "start_time": "2020-06-29T09:52:29.923699Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pandas.core.resample.DatetimeIndexResampler object at 0x00000235B93291C0>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r = df_r.resample('3min')\n",
    "r"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:29.954725Z",
     "start_time": "2020-06-29T09:52:29.938710Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:00:00</th>\n",
       "      <td>-18.557620</td>\n",
       "      <td>3.754986</td>\n",
       "      <td>-2.914837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:03:00</th>\n",
       "      <td>-9.301231</td>\n",
       "      <td>-20.132471</td>\n",
       "      <td>-4.392933</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:06:00</th>\n",
       "      <td>24.460419</td>\n",
       "      <td>-13.000934</td>\n",
       "      <td>-17.912736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:09:00</th>\n",
       "      <td>1.129074</td>\n",
       "      <td>18.148905</td>\n",
       "      <td>-5.606146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:12:00</th>\n",
       "      <td>-14.773186</td>\n",
       "      <td>14.762627</td>\n",
       "      <td>-4.806115</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:15:00</th>\n",
       "      <td>4.809390</td>\n",
       "      <td>-1.206336</td>\n",
       "      <td>0.853410</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             A          B          C\n",
       "2020-01-01 00:00:00 -18.557620   3.754986  -2.914837\n",
       "2020-01-01 00:03:00  -9.301231 -20.132471  -4.392933\n",
       "2020-01-01 00:06:00  24.460419 -13.000934 -17.912736\n",
       "2020-01-01 00:09:00   1.129074  18.148905  -5.606146\n",
       "2020-01-01 00:12:00 -14.773186  14.762627  -4.806115\n",
       "2020-01-01 00:15:00   4.809390  -1.206336   0.853410"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.sum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.002767Z",
     "start_time": "2020-06-29T09:52:29.955726Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "      <th>C</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-01-01</th>\n",
       "      <td>2.087516</td>\n",
       "      <td>-4.774063</td>\n",
       "      <td>-4.128187</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-02-03</th>\n",
       "      <td>-7.733344</td>\n",
       "      <td>2.783660</td>\n",
       "      <td>-6.666669</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-03-02</th>\n",
       "      <td>-1.143192</td>\n",
       "      <td>3.086895</td>\n",
       "      <td>-5.506985</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-04-01</th>\n",
       "      <td>6.043423</td>\n",
       "      <td>2.306306</td>\n",
       "      <td>8.212821</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-05-01</th>\n",
       "      <td>0.459925</td>\n",
       "      <td>1.590324</td>\n",
       "      <td>5.209677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-06-01</th>\n",
       "      <td>10.527052</td>\n",
       "      <td>-7.414759</td>\n",
       "      <td>-4.798573</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-07-01</th>\n",
       "      <td>-5.057341</td>\n",
       "      <td>4.021047</td>\n",
       "      <td>-1.573824</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                    A         B         C\n",
       "2020-01-01   2.087516 -4.774063 -4.128187\n",
       "2020-02-03  -7.733344  2.783660 -6.666669\n",
       "2020-03-02  -1.143192  3.086895 -5.506985\n",
       "2020-04-01   6.043423  2.306306  8.212821\n",
       "2020-05-01   0.459925  1.590324  5.209677\n",
       "2020-06-01  10.527052 -7.414759 -4.798573\n",
       "2020-07-01  -5.057341  4.021047 -1.573824"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_r2 = pd.DataFrame(np.random.randn(200, 3),index=pd.date_range('1/1/2020', freq='D', periods=200),\n",
    "                  columns=['A', 'B', 'C'])\n",
    "r = df_r2.resample('CBMS')\n",
    "r.sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 采样聚合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.018782Z",
     "start_time": "2020-06-29T09:52:30.003769Z"
    }
   },
   "outputs": [],
   "source": [
    "r = df_r.resample('3T')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.034796Z",
     "start_time": "2020-06-29T09:52:30.019783Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01 00:00:00   -0.103098\n",
       "2020-01-01 00:03:00   -0.051674\n",
       "2020-01-01 00:06:00    0.135891\n",
       "2020-01-01 00:09:00    0.006273\n",
       "2020-01-01 00:12:00   -0.082073\n",
       "2020-01-01 00:15:00    0.048094\n",
       "Freq: 3T, Name: A, dtype: float64"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r['A'].mean()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.050811Z",
     "start_time": "2020-06-29T09:52:30.035797Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sum</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:00:00</th>\n",
       "      <td>-18.557620</td>\n",
       "      <td>-0.103098</td>\n",
       "      <td>1.006979</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:03:00</th>\n",
       "      <td>-9.301231</td>\n",
       "      <td>-0.051674</td>\n",
       "      <td>1.058341</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:06:00</th>\n",
       "      <td>24.460419</td>\n",
       "      <td>0.135891</td>\n",
       "      <td>0.941624</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:09:00</th>\n",
       "      <td>1.129074</td>\n",
       "      <td>0.006273</td>\n",
       "      <td>0.918252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:12:00</th>\n",
       "      <td>-14.773186</td>\n",
       "      <td>-0.082073</td>\n",
       "      <td>0.913586</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:15:00</th>\n",
       "      <td>4.809390</td>\n",
       "      <td>0.048094</td>\n",
       "      <td>0.885248</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                           sum      mean       std\n",
       "2020-01-01 00:00:00 -18.557620 -0.103098  1.006979\n",
       "2020-01-01 00:03:00  -9.301231 -0.051674  1.058341\n",
       "2020-01-01 00:06:00  24.460419  0.135891  0.941624\n",
       "2020-01-01 00:09:00   1.129074  0.006273  0.918252\n",
       "2020-01-01 00:12:00 -14.773186 -0.082073  0.913586\n",
       "2020-01-01 00:15:00   4.809390  0.048094  0.885248"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r['A'].agg([np.sum, np.mean, np.std])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 类似地，可以使用函数/lambda表达式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.065824Z",
     "start_time": "2020-06-29T09:52:30.050811Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>A</th>\n",
       "      <th>B</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:00:00</th>\n",
       "      <td>-18.557620</td>\n",
       "      <td>4.765960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:03:00</th>\n",
       "      <td>-9.301231</td>\n",
       "      <td>5.053521</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:06:00</th>\n",
       "      <td>24.460419</td>\n",
       "      <td>5.142711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:09:00</th>\n",
       "      <td>1.129074</td>\n",
       "      <td>5.433368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:12:00</th>\n",
       "      <td>-14.773186</td>\n",
       "      <td>4.963697</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2020-01-01 00:15:00</th>\n",
       "      <td>4.809390</td>\n",
       "      <td>5.531560</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                             A         B\n",
       "2020-01-01 00:00:00 -18.557620  4.765960\n",
       "2020-01-01 00:03:00  -9.301231  5.053521\n",
       "2020-01-01 00:06:00  24.460419  5.142711\n",
       "2020-01-01 00:09:00   1.129074  5.433368\n",
       "2020-01-01 00:12:00 -14.773186  4.963697\n",
       "2020-01-01 00:15:00   4.809390  5.531560"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "r.agg({'A': np.sum,'B': lambda x: max(x)-min(x)})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. 采样组的迭代\n",
    "#### 采样组的迭代和groupby迭代完全类似，对于每一个组都可以分别做相应操作"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.080837Z",
     "start_time": "2020-06-29T09:52:30.066825Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Group:  2020-01-01 00:00:00\n",
      "---------------------------\n",
      "2020-01-01 00:00:00    0\n",
      "2020-01-01 00:30:00    1\n",
      "2020-01-01 00:31:00    2\n",
      "dtype: int64\n",
      "\n",
      "Group:  2020-01-01 01:00:00\n",
      "---------------------------\n",
      "2020-01-01 01:00:00    3\n",
      "dtype: int64\n",
      "\n",
      "Group:  2020-01-01 02:00:00\n",
      "---------------------------\n",
      "Series([], dtype: int64)\n",
      "\n",
      "Group:  2020-01-01 03:00:00\n",
      "---------------------------\n",
      "2020-01-01 03:00:00    4\n",
      "2020-01-01 03:05:00    5\n",
      "dtype: int64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "small = pd.Series(range(6),index=pd.to_datetime(['2020-01-01 00:00:00', '2020-01-01 00:30:00'\n",
    "                                                 , '2020-01-01 00:31:00','2020-01-01 01:00:00'\n",
    "                                                 ,'2020-01-01 03:00:00','2020-01-01 03:05:00']))\n",
    "resampled = small.resample('H')\n",
    "for name, group in resampled:\n",
    "    print(\"Group: \", name)\n",
    "    print(\"-\" * 27)\n",
    "    print(group, end=\"\\n\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 四、窗口函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 下面主要介绍pandas中两类主要的窗口(window)函数:rolling/expanding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.096852Z",
     "start_time": "2020-06-29T09:52:30.081838Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    1.776324\n",
       "2020-01-02    0.970563\n",
       "2020-01-03   -0.273771\n",
       "2020-01-04   -2.245556\n",
       "2020-01-05    1.222465\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = pd.Series(np.random.randn(1000),index=pd.date_range('1/1/2020', periods=1000))\n",
    "s.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Rolling\n",
    "#### （a）常用聚合\n",
    "#### 所谓rolling方法，就是规定一个窗口，它和groupby对象一样，本身不会进行操作，需要配合聚合函数才能计算结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.112866Z",
     "start_time": "2020-06-29T09:52:30.097853Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Rolling [window=50,center=False,axis=0]"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling(window=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.128880Z",
     "start_time": "2020-06-29T09:52:30.113867Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01         NaN\n",
       "2020-01-02         NaN\n",
       "2020-01-03         NaN\n",
       "2020-01-04         NaN\n",
       "2020-01-05         NaN\n",
       "                ...   \n",
       "2022-09-22   -0.109717\n",
       "2022-09-23   -0.121161\n",
       "2022-09-24   -0.108259\n",
       "2022-09-25   -0.116608\n",
       "2022-09-26   -0.146752\n",
       "Freq: D, Length: 1000, dtype: float64"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling(window=50).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### min_periods参数是指需要的非缺失数据点数量阀值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.144895Z",
     "start_time": "2020-06-29T09:52:30.129881Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01         NaN\n",
       "2020-01-02         NaN\n",
       "2020-01-03    0.366879\n",
       "2020-01-04    0.025182\n",
       "2020-01-05    0.057807\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling(window=50,min_periods=3).mean().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### count/sum/mean/median/min/max/std/var/skew/kurt/quantile/cov/corr都是常用的聚合函数\n",
    "#### （b）rolling的apply聚合\n",
    "#### 使用apply聚合时，只需记住传入的是window大小的Series，输出的必须是标量即可，比如如下计算变异系数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.415136Z",
     "start_time": "2020-06-29T09:52:30.145896Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01          NaN\n",
       "2020-01-02          NaN\n",
       "2020-01-03     4.833097\n",
       "2020-01-04    63.574687\n",
       "2020-01-05    24.017778\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling(window=50,min_periods=3).apply(lambda x:x.std()/x.mean()).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （c）基于时间的rolling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.430150Z",
     "start_time": "2020-06-29T09:52:30.416137Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    2.063499\n",
       "2020-01-02    0.294746\n",
       "2020-01-03    0.366879\n",
       "2020-01-04    0.025182\n",
       "2020-01-05    0.057807\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling('15D').mean().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 可选closed='right'（默认）\\'left'\\'both'\\'neither'参数，决定端点的包含情况"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.446164Z",
     "start_time": "2020-06-29T09:52:30.431151Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    2.063499\n",
       "2020-01-02    0.589492\n",
       "2020-01-03    1.100636\n",
       "2020-01-04    0.100730\n",
       "2020-01-05    0.289035\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling('15D', closed='right').sum().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Expanding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （a）expanding函数\n",
    "#### 普通的expanding函数等价与rolling(window=len(s),min_periods=1)，是对序列的累计计算"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以下这3个等价"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.461177Z",
     "start_time": "2020-06-29T09:52:30.447165Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    2.063499\n",
       "2020-01-02    0.589492\n",
       "2020-01-03    1.100636\n",
       "2020-01-04    0.100730\n",
       "2020-01-05    0.289035\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.rolling(window=len(s),min_periods=1).sum().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.477192Z",
     "start_time": "2020-06-29T09:52:30.462178Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    2.063499\n",
       "2020-01-02    0.589492\n",
       "2020-01-03    1.100636\n",
       "2020-01-04    0.100730\n",
       "2020-01-05    0.289035\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.expanding().sum().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### apply方法也是同样可用的"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.605306Z",
     "start_time": "2020-06-29T09:52:30.478193Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    2.063499\n",
       "2020-01-02    0.589492\n",
       "2020-01-03    1.100636\n",
       "2020-01-04    0.100730\n",
       "2020-01-05    0.289035\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.expanding().apply(lambda x:sum(x)).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （b）几个特别的Expanding类型函数\n",
    "#### cumsum/cumprod/cummax/cummin都是特殊expanding累计计算方法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.621320Z",
     "start_time": "2020-06-29T09:52:30.605306Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    2.063499\n",
       "2020-01-02    0.589492\n",
       "2020-01-03    1.100636\n",
       "2020-01-04    0.100730\n",
       "2020-01-05    0.289035\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.cumsum().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.637335Z",
     "start_time": "2020-06-29T09:52:30.622322Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01    1.776324\n",
       "2020-01-02    1.776324\n",
       "2020-01-03    1.776324\n",
       "2020-01-04    1.776324\n",
       "2020-01-05    1.776324\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.cummax().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### shift/diff/pct_change都是涉及到了元素关系\n",
    "#### ①shift是指序列索引不变，但值向后移动\n",
    "#### ②diff是指前后元素的差，period参数表示间隔，默认为1，并且可以为负\n",
    "#### ③pct_change是值前后元素的变化百分比，period参数与diff类似"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.653348Z",
     "start_time": "2020-06-29T09:52:30.638336Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01         NaN\n",
       "2020-01-02         NaN\n",
       "2020-01-03    2.063499\n",
       "2020-01-04   -1.474007\n",
       "2020-01-05    0.511144\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.shift(2).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.669363Z",
     "start_time": "2020-06-29T09:52:30.653348Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01         NaN\n",
       "2020-01-02         NaN\n",
       "2020-01-03         NaN\n",
       "2020-01-04   -3.063405\n",
       "2020-01-05    1.662312\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.diff(3).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.685377Z",
     "start_time": "2020-06-29T09:52:30.670364Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2020-01-01         NaN\n",
       "2020-01-02         NaN\n",
       "2020-01-03         NaN\n",
       "2020-01-04   -1.484568\n",
       "2020-01-05   -1.127750\n",
       "Freq: D, dtype: float64"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s.pct_change(3).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 五、问题与练习\n",
    "#### 【问题一】 如何对date_range进行批量加帧操作或对某一时间段加大时间戳密度？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 【问题二】 如何批量增加TimeStamp的精度？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.to_datetime('2020/1/1 00:00:00.123456789')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 【问题三】 对于超出处理时间的时间点，是否真的完全没有处理方法？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 【问题四】 给定一组非连续的日期，怎么快速找出位于其最大日期和最小日期之间，且没有出现在该组日期中的日期？"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 【练习一】 现有一份关于某超市牛奶销售额的时间序列数据，请完成下列问题：\n",
    "#### （a）销售额出现最大值的是星期几？（提示：利用dayofweek函数）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.701392Z",
     "start_time": "2020-06-29T09:52:30.686379Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>日期</th>\n",
       "      <th>销售额</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017/2/17</td>\n",
       "      <td>2154</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017/2/18</td>\n",
       "      <td>2095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017/2/19</td>\n",
       "      <td>3459</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017/2/20</td>\n",
       "      <td>2198</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017/2/21</td>\n",
       "      <td>2413</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          日期   销售额\n",
       "0  2017/2/17  2154\n",
       "1  2017/2/18  2095\n",
       "2  2017/2/19  3459\n",
       "3  2017/2/20  2198\n",
       "4  2017/2/21  2413"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv('data/time_series_one.csv').head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.716405Z",
     "start_time": "2020-06-29T09:52:30.702393Z"
    }
   },
   "outputs": [],
   "source": [
    "df = pd.read_csv('data/time_series_one.csv', parse_dates=['日期']) # 要加上这个parse_dates"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.731419Z",
     "start_time": "2020-06-29T09:52:30.717407Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "6"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 周日，注意dayofweek函数结果里0对应周一；6对应周日\n",
    "df['日期'].dt.dayofweek[df['销售额'].idxmax()]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （b）计算除去春节、国庆、五一节假日的月度销售总额"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.747433Z",
     "start_time": "2020-06-29T09:52:30.732420Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>销售额</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>日期</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-02-01</th>\n",
       "      <td>31740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-03-01</th>\n",
       "      <td>80000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-04-01</th>\n",
       "      <td>74734</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-05-01</th>\n",
       "      <td>76237</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-06-01</th>\n",
       "      <td>80750</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              销售额\n",
       "日期               \n",
       "2017-02-01  31740\n",
       "2017-03-01  80000\n",
       "2017-04-01  74734\n",
       "2017-05-01  76237\n",
       "2017-06-01  80750"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "holiday = pd.date_range(start='20170501', end='20170503').append(\n",
    "          pd.date_range(start='20171001', end='20171007')).append(\n",
    "          pd.date_range(start='20180215', end='20180221')).append(\n",
    "          pd.date_range(start='20180501', end='20180503')).append(\n",
    "          pd.date_range(start='20181001', end='20181007')).append(\n",
    "          pd.date_range(start='20190204', end='20190224')).append(\n",
    "          pd.date_range(start='20190501', end='20190503')).append(\n",
    "          pd.date_range(start='20191001', end='20191007'))\n",
    "result = df[~df['日期'].isin(holiday)].set_index('日期').resample('MS').sum()\n",
    "result.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （c）按季度计算周末（周六和周日）的销量总额"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.763447Z",
     "start_time": "2020-06-29T09:52:30.747433Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>销售额</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>日期</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-01-01</th>\n",
       "      <td>32894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-04-01</th>\n",
       "      <td>66692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-07-01</th>\n",
       "      <td>69099</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-10-01</th>\n",
       "      <td>70384</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-01</th>\n",
       "      <td>74671</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              销售额\n",
       "日期               \n",
       "2017-01-01  32894\n",
       "2017-04-01  66692\n",
       "2017-07-01  69099\n",
       "2017-10-01  70384\n",
       "2018-01-01  74671"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result = df[df['日期'].dt.dayofweek.isin([5,6])].set_index('日期').resample('QS').sum()\n",
    "result.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （d）从最后一天开始算起，跳过周六和周一，以5天为一个时间单位向前计算销售总和"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.779462Z",
     "start_time": "2020-06-29T09:52:30.764448Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>5天总额</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>日期</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2017-02-22</th>\n",
       "      <td>9855</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-03-01</th>\n",
       "      <td>12296</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-03-08</th>\n",
       "      <td>13323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-03-15</th>\n",
       "      <td>13845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2017-03-22</th>\n",
       "      <td>11356</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "             5天总额\n",
       "日期               \n",
       "2017-02-22   9855\n",
       "2017-03-01  12296\n",
       "2017-03-08  13323\n",
       "2017-03-15  13845\n",
       "2017-03-22  11356"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看不懂\n",
    "df_temp = df[~df['日期'].dt.dayofweek.isin([5,6])].set_index('日期').iloc[::-1]\n",
    "L_temp,date_temp = [],[0]*df_temp.shape[0]\n",
    "for i in range(df_temp.shape[0]//5):\n",
    "    L_temp.extend([i]*5)\n",
    "L_temp.extend([df_temp.shape[0]//5]*(df_temp.shape[0]-df_temp.shape[0]//5*5))\n",
    "date_temp = pd.Series([i%5==0 for i in range(df_temp.shape[0])])\n",
    "df_temp['num'] = L_temp\n",
    "result = pd.DataFrame({'5天总额':df_temp.groupby('num')['销售额'].sum().values},\n",
    "                       index=df_temp.reset_index()[date_temp]['日期']).iloc[::-1]\n",
    "result.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （e）假设现在发现数据有误，所有同一周里的周一与周五的销售额记录颠倒了，请计算2018年中每月第一个周一的销售额（如果该周没有周一或周五的记录就保持不动）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.811490Z",
     "start_time": "2020-06-29T09:52:30.780463Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>日期</th>\n",
       "      <th>销售额</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>318</th>\n",
       "      <td>2018-01-01</td>\n",
       "      <td>2863.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>353</th>\n",
       "      <td>2018-02-05</td>\n",
       "      <td>2321.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>2018-03-05</td>\n",
       "      <td>2705.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>409</th>\n",
       "      <td>2018-04-02</td>\n",
       "      <td>2487.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>444</th>\n",
       "      <td>2018-05-07</td>\n",
       "      <td>3204.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>472</th>\n",
       "      <td>2018-06-04</td>\n",
       "      <td>2927.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>500</th>\n",
       "      <td>2018-07-02</td>\n",
       "      <td>2574.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>535</th>\n",
       "      <td>2018-08-06</td>\n",
       "      <td>2504.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>563</th>\n",
       "      <td>2018-09-03</td>\n",
       "      <td>2483.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>591</th>\n",
       "      <td>2018-10-01</td>\n",
       "      <td>2431.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>626</th>\n",
       "      <td>2018-11-05</td>\n",
       "      <td>2395.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>654</th>\n",
       "      <td>2018-12-03</td>\n",
       "      <td>2373.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            日期     销售额\n",
       "318 2018-01-01  2863.0\n",
       "353 2018-02-05  2321.0\n",
       "381 2018-03-05  2705.0\n",
       "409 2018-04-02  2487.0\n",
       "444 2018-05-07  3204.0\n",
       "472 2018-06-04  2927.0\n",
       "500 2018-07-02  2574.0\n",
       "535 2018-08-06  2504.0\n",
       "563 2018-09-03  2483.0\n",
       "591 2018-10-01  2431.0\n",
       "626 2018-11-05  2395.0\n",
       "654 2018-12-03  2373.0"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 看不懂\n",
    "from datetime import datetime \n",
    "df_temp = df.copy()\n",
    "df_fri = df.shift(4)[df.shift(4)['日期'].dt.dayofweek==1]['销售额']\n",
    "df_mon = df.shift(-4)[df.shift(-4)['日期'].dt.dayofweek==5]['销售额']\n",
    "df_temp.loc[df_fri.index,['销售额']] = df_fri\n",
    "df_temp.loc[df_mon.index,['销售额']] = df_mon\n",
    "df_temp.loc[df_temp[df_temp['日期'].dt.year==2018]['日期'][\n",
    "        df_temp[df_temp['日期'].dt.year==2018]['日期'].apply(\n",
    "        lambda x:True if datetime.strptime(str(x).split()[0],'%Y-%m-%d').weekday() == 0 \n",
    "        and 1 <= datetime.strptime(str(x).split()[0],'%Y-%m-%d').day <= 7 else False)].index,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 【练习二】 继续使用上一题的数据，请完成下列问题：\n",
    "#### （a）以50天为窗口计算滑窗均值和滑窗最大值（min_periods设为1）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.827505Z",
     "start_time": "2020-06-29T09:52:30.812492Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "日期\n",
       "2017-02-17    2154.000000\n",
       "2017-02-18    2124.500000\n",
       "2017-02-19    2569.333333\n",
       "2017-02-20    2476.500000\n",
       "2017-02-21    2463.800000\n",
       "Name: 销售额, dtype: float64"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('data/time_series_one.csv',index_col='日期',parse_dates=['日期'])\n",
    "df['销售额'].rolling(window=50,min_periods=1).mean().head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.843519Z",
     "start_time": "2020-06-29T09:52:30.828506Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "日期\n",
       "2017-02-17    2154.0\n",
       "2017-02-18    2154.0\n",
       "2017-02-19    3459.0\n",
       "2017-02-20    3459.0\n",
       "2017-02-21    3459.0\n",
       "Name: 销售额, dtype: float64"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['销售额'].rolling(window=50,min_periods=1).max().head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （b）现在有如下规则：若当天销售额超过向前5天的均值，则记为1，否则记为0，请给出2018年相应的计算结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:30.971633Z",
     "start_time": "2020-06-29T09:52:30.843519Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>销售额</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-01-01</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-02</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-03</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-04</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-05</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            销售额\n",
       "2018-01-01  1.0\n",
       "2018-01-02  0.0\n",
       "2018-01-03  0.0\n",
       "2018-01-04  0.0\n",
       "2018-01-05  0.0"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def f(x):\n",
    "    if len(x) == 6:\n",
    "        return 1 if x[-1]>np.mean(x[:-1]) else 0\n",
    "    else:\n",
    "        return 0\n",
    "result_b = df.loc[pd.date_range(start='20171227',end='20181231'),:].rolling(\n",
    "                                                    window=6,min_periods=1).agg(f)[5:].head()\n",
    "result_b.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### （c）将(c)中的“向前5天”改为“向前非周末5天”，请再次计算结果"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-06-29T09:52:31.271902Z",
     "start_time": "2020-06-29T09:52:30.971633Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>销售额</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2018-01-01</th>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-02</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-03</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-04</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018-01-05</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            销售额\n",
       "2018-01-01  1.0\n",
       "2018-01-02  0.0\n",
       "2018-01-03  0.0\n",
       "2018-01-04  0.0\n",
       "2018-01-05  0.0"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def f(x):\n",
    "    if len(x) == 8:\n",
    "        return 1 if x[-1]>np.mean(x[:-1][pd.Series([\n",
    "            False if i in [5,6] else True for i in x[:-1].index.dayofweek],index=x[:-1].index)]) else 0\n",
    "    else:\n",
    "        return 0\n",
    "result_c = df.loc[pd.date_range(start='20171225',end='20181231'),:].rolling(\n",
    "                                    window=8,min_periods=1).agg(f)[7:].head()\n",
    "result_c.head()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "joyfulpd",
   "language": "python",
   "name": "joyfulpd"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}