the portable Python dataframe library

Lightweight and extensible compatibility layer between dataframe libraries!

A bioinformatics extension of 🤗 Datasets library, built for ML applications on biological and omics data, offering easy integration of metadata and low-code data management tools.

poor man´s data lake - Simple api to efficiently query your parquet datasets using Duckdb or polars

FastFlight is a high-performance data transfer framework using Apache Arrow Flight for efficient, modular, and pluggable data streaming with optional FastAPI integration for HTTP-based access.

Seamlessly switch Pandas DataFrame backend to PyArrow.

An open-source tool for reading OvertureMaps data with multiprocessing and additional Quality-of-Life features

Type annotations for pyarrow

Work with bioinformatic files using Arrow, Polars, and/or DuckDB

Maps of Sweden in GeoParquet

A loose implementation of the deltalake protocol, written in Python on top of pyarrow, focused on extensibility, customizability, and distributed data.

Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀

manylinux2014 Python pkg builds

Performance comparison between Ray and PyTorch, across different Machine Learning and Big Data related operations. It is developed as a project for the NTUA course Analysis and Design of Information Systems.

Pyarrow Tutorials

Python scripts to download, process, and analyze the New York City Taxi and Limousine Commission (TLC) Trip Record Data dataset

MongoDB tutorial repository

View Apache Parquet Files In Your Terminal

Python scripts to process, and analyze log files using PySpark.

Command-line interface to quickly generate fake CSV and JSON data