Skip to content

Pandas

Jump to bottom
verthais edited this page Apr 13, 2023 · 3 revisions

Pandas

It is a library containing high-level data structures and tools that have been created to assist a Python programmer to perform powerful data manipulations, and discover information in that data in a simple and fast way.

  • High performance array and table structures for representation of homogenous and heterogeneous data sets: the Series and DataFrame objects
  • Flexible reshaping of data structure, allowing the ability to insert and delete both rows and columns of tabular data
  • Hierarchical indexing of data along multiple axes (both rows and columns), allowing multiple labels per data item
  • Labeling of series and tabular data to facilitate indexing and automatic alignment of data
  • Ability to easily identify and fix missing data, both in floating point and as non-floating point formats
  • Powerful grouping capabilities and a functionality to perform split-apply-combine operations on series and tabular data
  • Simple conversion from ragged and differently indexed data of both NumPy and Python data structures to pandas objects
  • Smart label-based slicing and subsetting of data sets, including intuitive and flexible merging, and joining of data with SQL-like constructs
  • Extensive I/O facilities to load and save data from multiple formats including CSV, Excel, relational and non-relational databases, HDF5 format, and JSON
  • Explicit support for time series-specific functionality, providing functionality for date range generation, moving window statistics, time shifting, lagging, and so on
  • Built-in support to retrieve and automatically parse data from various web-based data sources such as Yahoo!, Google Finance, the World Bank, and several others

pandas objects

A programmer of pandas will spend most of their time using two primary objects provided by the pandas framework: Series and DataFrame. The DataFrame objects will be the overall workhorse of pandas and the most frequently used as they provide the means to manipulate tabular and heterogeneous data.

Series object

The base data structure of pandas is the Series object, which is designed to operate similar to a NumPy array but also adds index capabilities. The result of an arithmetic operation (+, -, /, *, …) on two Series objects that are non-scalar values returns another Series object.

DataFrame object

In a way it is analogous to a database table in that it contains one or more columns of data of heterogeneous type (but a single type for all items in each respective column). There is a subtle difference in a DataFrame object as compared to a Series object. Passing a list to the [] operator of DataFrame retrieves the specified columns, whereas Series uses it as index labels to retrieve rows. Entire rows from a DataFrame can be retrieved using its .loc and .iloc properties.

Loading data from files and the Web

Loading DataFrame from CSV

   df = pd.read_csv('data/test1.csv', parse_dates=['date'], index_col='date')

Using the index_col parameter of the pd.read_csv() method to specify which column in the file should be used as the index. We can use the parse_dates parameter of the pd.read_csv() function to guide pandas on how to the content of the 'date' column into actual TimeStamp objects.

Loading data from the Web

   # imports for reading data from Yahoo!
   from pandas.io.data import DataReader
   from datetime import date
   from dateutil.relativedelta import relativedelta

   # read the last three months of data for GOOG
   goog = DataReader("GOOG",  "yahoo", 
                     date.today() + 
                     relativedelta(months=-3))

Home

Clone this wiki locally