Python client to Cloudera Impala.
Rationale and vision
Impala allows you to rapidly analyze large, distributed data sets. But it doesn't integrate easily with your ad hoc (Python) analytical tools (pandas, scikit-learn). impyla aims to remedy this.
This package offers:
pip-installable package for Impala-driven analytics anywhere
Eventually, we'll also support:
Integration with Spark
Running Python UDFs by compiling them to LLVM IR
For the latest (dev) version, clone the repo:
git clone git://github.com/laserson/impyla.git cd impyla python setup.py install
pip install impyla will install version 0.7 of this package,
which very different. We suggest you either clone or download the repo from
the GitHub page.
Impyla implements the Python DB API 2 (PEP 249)
import impala.dbapi conn = impala.dbapi.connect(host='my.host.com', port=21050) cursor = conn.cursor() cursor.execute('SELECT * FROM table LIMIT 100') for row in cursor: process(row)
You can also get back a pandas DataFrame object
import impala.util df = impala.util.as_pandas(cur) # carry df through scikit-learn, for example