Skip to content
A Python wrapper for MADlib( - an open source library for scalable in-database machine learning algorithms
Branch: master
Clone or download
Type Name Latest commit message Commit time
Failed to load latest commit information.
pymadlib extension of pandas support to more algorithms May 21, 2014
.gitignore Adding gitignore, cleaning up build files May 17, 2014
LICENSE.txt hello world Feb 23, 2013 hello world Feb 23, 2013 Plugging in domain name change (pivotalsoftware) Jul 29, 2014
README.txt hello world Feb 23, 2013 Replace old junk with pandas May 21, 2014

A Python wrapper for MADlib - an open source library for scalable in-database machine learning algorithms. You can visit PyMADlib's webpage for installation and usage tutorials.


PyMADlib currently has wrappers for the following algorithms in MADlib (version 0.5).

  1. Linear regression
  2. Logistic Regression
  3. SVM (regression & classification)
  4. K-Means
  5. LDA

Refer MADlib User Docs for MADlib's user documentation. Please note that PyMADlib as of now is only compatible with MADlib v0.5. You can obtain MADlib v0.5 from MADlib v0.5. We might add support to more recent versions of MADlib depending on adoption rate. Please email me if you have a strong case for an upgrade.


  1. You'll need the python extension psycopg2 to use PyMADlib.
  2. If you have matplotlib installed, you'll see Matplotlib visualizations for Linear Regression demo.
  3. If you have installed networkx, you'll see a visualization of the k-means demo
  4. PyROC is included in the source of this distribution with permission from its developer. You'll see a visualization of the ROC curves for Logistic Regression.


To configure your DB Connection parameters You should create a file in your home directory


that should look like so :

user = gpadmin  
password = XXXXX  
hostname = (or the IP of your DB server)  
port = 5432 (the port# of your DB)  
database = vatsandb (the database you wish to connect to)  

Installation Instructions

PyMADlib depends on MADlib, psycopg2 and Pandas. It is easiest to work with PyMADlib if you have Anaconda Python.

Build Environment Setup on Mac OS X 10.8

vatsan-mac$ which python
vatsan-mac$ which conda
  • If you haven't installed PostgreSQL on your Mac already, you'll have to download & install PostGreSQL for Mac. This is so that we get some required libraries to compile the SQL Engine: psycopg2. The easiest way to install PostGreSQL on Mac is via Once you've downloaded and installed PostGreSQL on Mac, it should typically be found under /Library/PostgreSQL
vatsan-mac$ ls /Library/PostgreSQL/9.2/
Library include
bin installer scripts
data lib share

I don't think the version of the PostGreSQL matters (9.1 or above is fine).

  • You may need to create some symlinks to libpq & libssl so that psycopg2 is able to find it:
vatsan-mac$ sudo ln -s /Users/vatsan/anaconda/lib/libssl.1.0.0.dylib /usr/lib
vatsan-mac$ sudo ln -s /Users/vatsan/anaconda/lib/libcrypto.1.0.0.dylib /usr/lib
  • Install Psycopg2
vatsan-mac$ conda install distribute
vatsan-mac$ pip install psycopg2
  • Now we're ready to test if the installations of the required libraries were successful.
vatsan-mac$ python -c 'import psycopg2'

If the above command did not error out, then installation was successful.

  • You may install PyMADlib by downloading the source (from PyPI) and then run the following
sudo python build
sudo python install
  • If you use easy_install or pip, simply run :
sudo easy_install pymadlib

Usage Tutorial

Visit PyMADlib Tutorial for a tutorial on using PyMADlib Also visit PyMADlib IPython NB to download the IPython NB tutorial

Running the Demos

You may run the demo from the extracted directory of pymadlib like so :


If you installed PyMADlib using instructions in the previous section, then simply run

python -c 'from pymadlib.example import runDemos; runDemos()'

Remember to close the Matplotlib windows that pop-up to continue with the rest of the demo.


K-Means Cluster Visualization

Scatter Plot - Linear Regression (numeric attributes only)

Scatter Plot - Linear Regression (with categorical attributes)

ROC Curve - Logistic Regression

Random graph visualization - Networkx

Datasets packaged with this installation

PyMADlib packages publicly available datasets from the UCI machine learning repository and other sources.

  1. Wine quality dataset from UCI Machine Learning repository
  2. Auto MPG dataset from UCI ML repository from UCI Machine Learning repository
  3. Wine quality dataset from UCI Machine Learning repository
  4. Obama-Romney second presidential debate (2012) transcripts


You can’t perform that action at this time.