From ad59b22fce5a1e632f6509c963e73a524847f458 Mon Sep 17 00:00:00 2001 From: Julia Evans Date: Fri, 3 Jan 2014 16:51:23 -0500 Subject: [PATCH] Use to_datetime() in Unix timestamp example instead of .astype('datetime64'). Fixes #2. --- ...pter 8 - How to deal with timestamps.ipynb | 170 ++++-------------- 1 file changed, 32 insertions(+), 138 deletions(-) diff --git a/cookbook/Chapter 8 - How to deal with timestamps.ipynb b/cookbook/Chapter 8 - How to deal with timestamps.ipynb index 8f127c68..5dae1eb0 100644 --- a/cookbook/Chapter 8 - How to deal with timestamps.ipynb +++ b/cookbook/Chapter 8 - How to deal with timestamps.ipynb @@ -166,17 +166,15 @@ "source": [ "Every numpy array and pandas series has a dtype -- this is usually `int64`, `float64`, or `object`. Some of the time types available are `datetime64[s]`, `datetime64[ms]`, and `datetime64[us]`. There are also `timedelta` types, similarly.\n", "\n", - "Normally you can convert between dtypes using the `astype()` method, but for some reason that throws an error for me right now.\n", - "\n", - "Changing the dtype manually seems to work:" + "We can use the `pd.to_datetime` function to convert our integer timestamps into datetimes. This is a constant-time operation -- we're not actually changing any of the data, just how pandas thinks about it." ] }, { "cell_type": "code", "collapsed": false, "input": [ - "popcon['atime'].dtype = dtype('datetime64[s]')\n", - "popcon['ctime'].dtype = dtype('datetime64[s]')" + "popcon['atime'] = pd.to_datetime(popcon['atime'], unit='s')\n", + "popcon['ctime'] = pd.to_datetime(popcon['ctime'], unit='s')" ], "language": "python", "metadata": {}, @@ -187,87 +185,24 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Huh, this is weird! The dates still aren't showing up properly." + "If we look at the dtype now, it's `\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
atimectimepackage-namemru-programtag
0 1387295797 1367633260 perl-base /usr/bin/perl NaN
1 1387295796 1354370480 login /bin/su NaN
2 1387295743 1354341275 libtalloc2 /usr/lib/x86_64-linux-gnu/libtalloc.so.2.0.7 NaN
3 1387295743 1387224204 libwbclient0 /usr/lib/x86_64-linux-gnu/libwbclient.so.0 <RECENT-CTIME>
4 1387295742 1354341253 libselinux1 /lib/x86_64-linux-gnu/libselinux.so.1 NaN
\n", - "" - ], "metadata": {}, "output_type": "pyout", "prompt_number": 6, "text": [ - " atime ctime package-name mru-program tag\n", - "0 1387295797 1367633260 perl-base /usr/bin/perl NaN\n", - "1 1387295796 1354370480 login /bin/su NaN\n", - "2 1387295743 1354341275 libtalloc2 /usr/lib/x86_64-linux-gnu/libtalloc.so.2.0.7 NaN\n", - "3 1387295743 1387224204 libwbclient0 /usr/lib/x86_64-linux-gnu/libwbclient.so.0 \n", - "4 1387295742 1354341253 libselinux1 /lib/x86_64-linux-gnu/libselinux.so.1 NaN" + "dtype('0\n", " 2013-12-17 15:56:37\n", " 2013-05-04 02:07:40\n", - " perl-base\n", - " /usr/bin/perl\n", + " perl-base\n", + " /usr/bin/perl\n", " NaN\n", " \n", " \n", " 1\n", " 2013-12-17 15:56:36\n", " 2012-12-01 14:01:20\n", - " login\n", - " /bin/su\n", + " login\n", + " /bin/su\n", " NaN\n", " \n", " \n", " 2\n", " 2013-12-17 15:55:43\n", " 2012-12-01 05:54:35\n", - " libtalloc2\n", - " /usr/lib/x86_64-linux-gnu/libtalloc.so.2.0.7\n", + " libtalloc2\n", + " /usr/lib/x86_64-linux-gnu/libtalloc.so.2.0.7\n", " NaN\n", " \n", " \n", " 3\n", " 2013-12-17 15:55:43\n", " 2013-12-16 20:03:24\n", - " libwbclient0\n", - " /usr/lib/x86_64-linux-gnu/libwbclient.so.0\n", + " libwbclient0\n", + " /usr/lib/x86_64-linux-gnu/libwbclient.so.0\n", " <RECENT-CTIME>\n", " \n", " \n", " 4\n", " 2013-12-17 15:55:42\n", " 2012-12-01 05:54:13\n", - " libselinux1\n", - " /lib/x86_64-linux-gnu/libselinux.so.1\n", - " NaN\n", - " \n", - " \n", - " 5\n", - " 2013-12-17 15:55:42\n", - " 2012-12-01 05:54:35\n", - " libstdc++6\n", - " /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16\n", - " NaN\n", - " \n", - " \n", - " 6\n", - " 2013-12-17 15:55:40\n", - " 2013-12-16 20:03:22\n", - " libpam-winbind\n", - " /lib/x86_64-linux-gnu/security/pam_winbind.so\n", - " <RECENT-CTIME>\n", - " \n", - " \n", - " 7\n", - " 2013-12-17 15:55:40\n", - " 2012-12-01 05:54:13\n", - " libpam-modules\n", - " /lib/x86_64-linux-gnu/security/pam_unix.so\n", - " NaN\n", - " \n", - " \n", - " 8\n", - " 2013-12-17 15:55:40\n", - " 2012-12-01 05:54:13\n", - " libpam-ck-connector\n", - " /lib/security/pam_ck_connector.so\n", - " NaN\n", - " \n", - " \n", - " 9\n", - " 2013-12-17 15:55:40\n", - " 2012-12-01 05:54:13\n", - " libpam-cap\n", - " /lib/x86_64-linux-gnu/security/pam_cap.so\n", + " libselinux1\n", + " /lib/x86_64-linux-gnu/libselinux.so.1\n", " NaN\n", " \n", " \n", @@ -395,17 +287,12 @@ "output_type": "pyout", "prompt_number": 7, "text": [ - " atime ctime package-name mru-program tag\n", - "0 2013-12-17 15:56:37 2013-05-04 02:07:40 perl-base /usr/bin/perl NaN\n", - "1 2013-12-17 15:56:36 2012-12-01 14:01:20 login /bin/su NaN\n", - "2 2013-12-17 15:55:43 2012-12-01 05:54:35 libtalloc2 /usr/lib/x86_64-linux-gnu/libtalloc.so.2.0.7 NaN\n", - "3 2013-12-17 15:55:43 2013-12-16 20:03:24 libwbclient0 /usr/lib/x86_64-linux-gnu/libwbclient.so.0 \n", - "4 2013-12-17 15:55:42 2012-12-01 05:54:13 libselinux1 /lib/x86_64-linux-gnu/libselinux.so.1 NaN\n", - "5 2013-12-17 15:55:42 2012-12-01 05:54:35 libstdc++6 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.16 NaN\n", - "6 2013-12-17 15:55:40 2013-12-16 20:03:22 libpam-winbind /lib/x86_64-linux-gnu/security/pam_winbind.so \n", - "7 2013-12-17 15:55:40 2012-12-01 05:54:13 libpam-modules /lib/x86_64-linux-gnu/security/pam_unix.so NaN\n", - "8 2013-12-17 15:55:40 2012-12-01 05:54:13 libpam-ck-connector /lib/security/pam_ck_connector.so NaN\n", - "9 2013-12-17 15:55:40 2012-12-01 05:54:13 libpam-cap /lib/x86_64-linux-gnu/security/pam_cap.so NaN" + " atime ctime package-name mru-program tag\n", + "0 2013-12-17 15:56:37 2013-05-04 02:07:40 perl-base /usr/bin/perl NaN\n", + "1 2013-12-17 15:56:36 2012-12-01 14:01:20 login /bin/su NaN\n", + "2 2013-12-17 15:55:43 2012-12-01 05:54:35 libtalloc2 /usr/lib/x86_64-linux-gnu/libtalloc.so.2.0.7 NaN\n", + "3 2013-12-17 15:55:43 2013-12-16 20:03:24 libwbclient0 /usr/lib/x86_64-linux-gnu/libwbclient.so.0 \n", + "4 2013-12-17 15:55:42 2012-12-01 05:54:13 libselinux1 /lib/x86_64-linux-gnu/libselinux.so.1 NaN" ] } ], @@ -415,7 +302,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Okay, great! So now I can just look at packages that aren't libraries!\n", + "Now suppose we want to look at all packages that aren't libraries.\n", "\n", "First, I want to get rid of everything with timestamp 0. Notice how we can just use a string in this comparison, even though it's actually a timestamp on the inside? That is because pandas is amazing." ] @@ -431,6 +318,13 @@ "outputs": [], "prompt_number": 8 }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we can use pandas' magical string abilities to just look at rows where the package name doesn't contain 'lib'." + ] + }, { "cell_type": "code", "collapsed": false,