Best-of Python
This curated list contains 390 awesome open-source projects with a total of 1.4M stars grouped into 28 categories. All projects are ranked by a project-quality score, which is calculated based on various metrics automatically collected from GitHub and different package managers. If you like to add or update projects, feel free to open an issue, submit a pull request, or directly edit the projects.yaml. Contributions are very welcome!
Contents
- Data Serialization 16 projects
- Data Containers & Dataframes 30 projects
- Data Structures 15 projects
- Data Validation 15 projects
- Algorithms & Design Patterns 4 projects
- Date & Time Utilities 9 projects
- File & Path Utilities 10 projects
- Compatiblity 7 projects
- Cryptography 7 projects
- Infrastructure & DevOps 19 projects
- Process Utilities 4 projects
- Asynchronous Programming 7 projects
- Configuration 9 projects
- CLI Development 18 projects
- Development Tools 1 projects
- Data Caching 6 projects
- GUI Development 10 projects
- Computer & Machine Vision 1 projects
- Machine Learning & Data Engineering 1 projects
- Text Data 12 projects
- Web Development 1 projects
- Database Clients 64 projects
- Data Loading & Extraction 30 projects
- Data Pipelines & Streaming 43 projects
- File Formats 3 projects
- Code Inspection 4 projects
- General Utilities 15 projects
- Python Implementations 6 projects
- Others 21 projects
Explanation
🥇 🥈 🥉 Combined project-quality score⭐️ Star count from GitHub🐣 New project (less than 6 months old)💤 Inactive project (6 months no activity)💀 Dead project (12 months no activity)📈 📉 Project is trending up or down➕ Project was recently added❗️ Warning (e.g. missing/risky license)👨💻 Contributors count from GitHub🔀 Fork count from GitHub📋 Issue count from GitHub⏱️ Last update timestamp on package manager📥 Download count from package manager📦 Number of dependent projectsPandas related project
Data Serialization
protobuf (🥇 48 · ⭐ 56K) - Protocol Buffers - Googles data interchange format. BSD-3
-
GitHub (
👨💻 1K ·🔀 14K ·📥 37M ·📦 250K ·📋 5.1K - 15% open ·⏱️ 17.08.2022):git clone https://github.com/protocolbuffers/protobuf -
PyPi (
📥 93M / month ·📦 15K ·⏱️ 24.06.2022):pip install protobuf -
Conda (
📥 9.6M ·⏱️ 10.05.2022):conda install -c conda-forge protobuf -
npm (
📥 4.5M / month ·📦 2.8K ·⏱️ 28.07.2022):npm install google-protobuf
flatbuffers (🥇 40 · ⭐ 19K) - FlatBuffers: Memory Efficient Serialization Library. Apache-2
-
GitHub (
👨💻 580 ·🔀 2.8K ·📥 80K ·📦 2.8K ·📋 2K - 6% open ·⏱️ 17.08.2022):git clone https://github.com/google/flatbuffers -
PyPi (
📥 7.7M / month ·📦 210 ·⏱️ 10.05.2021):pip install flatbuffers -
Conda (
📥 410K ·⏱️ 04.03.2022):conda install -c conda-forge flatbuffers -
npm (
📥 410K / month ·📦 220 ·⏱️ 25.02.2022):npm install flatbuffers
marshmallow (🥈 38 · ⭐ 6.2K) - A lightweight library for converting complex objects to and from.. MIT
ultrajson (🥈 36 · ⭐ 3.8K) - Ultra fast JSON decoder and encoder written in C with Python bindings. BSD-3
orjson (🥈 35 · ⭐ 3.6K) - Fast, correct Python JSON library supporting dataclasses, datetimes,.. Apache-2
simplejson (🥈 34 · ⭐ 1.5K) - simplejson is a simple, fast, extensible JSON encoder/decoder for.. MIT
cloudpickle (🥈 34 · ⭐ 1.2K) - Extended pickling support for Python objects. BSD-3
msgpack (🥉 33 · ⭐ 1.6K) - MessagePack serializer implementation for Python msgpack.org[Python]. Apache-2
jsonpickle (🥉 33 · ⭐ 1K) - Python library for serializing any arbitrary object graph into JSON... BSD-3
python-rapidjson (🥉 27 · ⭐ 460) - Python wrapper around rapidjson. MIT
pysimdjson (🥉 26 · ⭐ 550) - Python bindings for the simdjson project. MIT
Data Containers & Dataframes
General-purpose data containers as well as utilities & extensions for pandas.
h5py (🥇 41 · ⭐ 1.8K) - HDF5 for Python -- The h5py package is a Pythonic interface to the HDF5.. BSD-3
numexpr (🥈 33 · ⭐ 1.8K) - Fast numerical array expression evaluator for Python, NumPy, PyTables,.. MIT
Bottleneck (🥈 33 · ⭐ 780) - Fast NumPy array functions written in C. BSD-2
Vaex (🥉 30 · ⭐ 7.2K) - Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization.. MIT
Pandaral·lel (🥉 28 · ⭐ 2.4K) - A simple and efficient tool to parallelize Pandas.. BSD-3 jupyter
datasketch (🥉 28 · ⭐ 1.8K) - MinHash, LSH, LSH Forest, Weighted MinHash, HyperLogLog,.. MIT
StaticFrame (🥉 26 · ⭐ 310) - Immutable and grow-only Pandas-like DataFrames with a more explicit.. MIT
Pandas Summary (🥉 20 · ⭐ 430) - A library for managing, validating, summarizing, and.. Apache-2
Show 8 hidden projects...
- Blaze (
🥈 31 ·⭐ 3.1K ·💀 ) - NumPy and Pandas interface to Big Data.BSD-3 - sklearn-pandas (
🥉 29 ·⭐ 2.6K) - Pandas integration with sklearn.❗️Zlibsklearn - Arctic (
🥉 28 ·⭐ 2.8K) - Arctic is a high performance datastore for numeric data.❗️LGPL-2.1 - pandasql (
🥉 27 ·⭐ 1.1K ·💀 ) - sqldf for pandas.MIT - bcolz (
🥉 27 ·⭐ 940 ·💀 ) - A columnar data container that can be compressed.BSD-3 - pickleDB (
🥉 22 ·⭐ 690 ·💀 ) - pickleDB is an open source key-value store using Pythons json module.BSD-3 - fletcher (
🥉 19 ·⭐ 220 ·💀 ) - Pandas ExtensionDType/Array backed by Apache Arrow.MIT - Bounter (
🥉 18 ·⭐ 940 ·💀 ) - Efficient Counter that uses a limited (bounded) amount of memory..MIT
Data Structures
pyrsistent (🥇 34 · ⭐ 1.7K) - Persistent/Immutable/Functional data structures for Python. MIT
python-sortedcontainers (🥈 29 · ⭐ 2.7K) - Python Sorted Container Types: Sorted List, Sorted.. Apache-2
python-benedict (🥈 28 · ⭐ 420) - dict subclass with keylist/keypath support, normalized I/O.. MIT
immutables (🥉 27 · ⭐ 960) - A high-performance immutable mapping type for Python. Apache-2
sqlitedict (🥉 27 · ⭐ 870) - Persistent dict, backed by sqlite3 and pickle, multithread-safe. Apache-2
python-box (🥉 26 · ⭐ 2K) - Python dictionaries with advanced dot notation access. MIT
ordered-set (🥉 26 · ⭐ 160 · 💤 ) - A mutable set that remembers the order of its entries. One of.. MIT
Show 4 hidden projects...
- addict (
🥈 28 ·⭐ 2.3K ·💀 ) - The Python Dict thats better than heroin.MIT - munch (
🥉 27 ·⭐ 620 ·💀 ) - A Munch is a Python dictionary that provides attribute-style access (a..MIT - anytree (
🥉 26 ·⭐ 730 ·💀 ) - Python tree data library.Apache-2 - cleverdict (
🥉 14 ·⭐ 88) - A JSON-friendly data structure which allows both object attributes and..MIT
Data Validation
jsonschema (🥇 40 · ⭐ 3.8K) - An implementation of the JSON Schema specification for Python. MIT
voluptuous (🥈 32 · ⭐ 1.7K) - CONTRIBUTIONS ONLY: Voluptuous, despite the name, is a Python data.. BSD-3
schematics (🥈 31 · ⭐ 2.5K · 💤 ) - Python Data Structures for Humans. BSD-3
validators (🥉 30 · ⭐ 610) - Python Data Validation for Humans. MIT
strictyaml (🥉 27 · ⭐ 1.1K · 💤 ) - Type-safe YAML parser and validator. MIT
dirty-equals (🥉 16 · ⭐ 500) - Doing dirty (but extremely useful) things with equals. MIT
Show 4 hidden projects...
- cerberus (
🥈 32 ·⭐ 2.8K ·💀 ) - Lightweight, extensible data validation library for Python.ISC - python-email-validator (
🥉 25 ·⭐ 620) - A robust email syntax and deliverability validation..❗️CC0-1.0 - valideer (
🥉 20 ·⭐ 260 ·💀 ) - Lightweight data validation and adaptation Python library.MIT - dataklasses (
🥉 7 ·⭐ 740 ·💤 ) - A different spin on dataclasses.❗Unlicensed
Algorithms & Design Patterns
transitions (🥇 30 · ⭐ 4.5K) - A lightweight, object-oriented finite state machine implementation.. MIT
algorithms (🥉 29 · ⭐ 21K) - Minimal examples of data structures and algorithms in Python. MIT
Show 1 hidden projects...
Date & Time Utilities
python-dateutil (🥇 38 · ⭐ 1.9K) - Useful extensions to the standard Python datetime features. Apache-2
dateparser (🥈 34 · ⭐ 2.1K) - python parser for human readable dates. BSD-3
parsedatetime (🥉 27 · ⭐ 650 · 💤 ) - Parse human-readable date/time strings. Apache-2
File & Path Utilities
filesystem_spec (🥇 37 · ⭐ 460) - A specification that python filesystems should adhere to. BSD-3
pyfilesystem2 (🥈 32 · ⭐ 1.7K) - Pythons Filesystem abstraction layer. MIT
appdirs (🥈 31 · ⭐ 900) - A small Python module for determining appropriate platform-specific dirs,.. MIT
scandir (🥉 29 · ⭐ 500 · 💤 ) - Better directory iterator and faster os.walk(), now in the Python.. BSD-3
Show 2 hidden projects...
Compatiblity
typing (🥈 33 · ⭐ 1.3K) - Python static typing home. Hosts the documentation and a user help.. Python-2.0
dataclasses (🥉 27 · ⭐ 550) - A backport of the dataclasses module for Python 3.6. Apache-2
futures (🥉 27 · ⭐ 220 · 💤 ) - Backport of the concurrent.futures package to Python 2.6 and 2.7. Python-2.0
Show 2 hidden projects...
- pathlib2 (
🥉 27 ·⭐ 69) - Backport of pathlib aiming to support the full stdlib Python API.MIT - contextlib2 (
🥉 26 ·⭐ 34) - contextlib2 is a backport of the standard librarys contextlib..❗️psfrag
Cryptography
cryptography (🥇 43 · ⭐ 5K) - cryptography is a package designed to expose cryptographic.. BSD-3
pycryptodomex (🥈 37 · ⭐ 2.1K) - A self-contained cryptographic library for Python. BSD-3
asn1crypto (🥉 34 · ⭐ 270) - Python ASN.1 library with a focus on performance and a pythonic API. MIT
Infrastructure & DevOps
ansible (🥇 48 · ⭐ 54K) - Ansible is a radically simple IT automation platform that makes your.. ❗️GPL-3.0
docker-compose (🥈 43 · ⭐ 27K) - Define and run multi-container applications with Docker. Apache-2
pulumi (🥈 40 · ⭐ 13K) - Pulumi - Universal Infrastructure as Code. Your Cloud, Your Language,.. Apache-2
paramiko (🥈 40 · ⭐ 7.8K · 📉 ) - The leading native Python SSHv2 protocol library. ❗️LGPL-2.1
kubernetes (🥈 38 · ⭐ 5K) - Official Python client library for kubernetes. Apache-2
pyinfra (🥉 30 · ⭐ 1.6K) - pyinfra automates infrastructure super fast at massive scale. It can be.. MIT
pypyr (🥉 21 · ⭐ 430) - pypyr task-runner cli & api for automation pipelines. Automate anything.. Apache-2
Show 5 hidden projects...
- sshtunnel (
🥉 29 ·⭐ 1K ·💀 ) - SSH tunnels to remote server.MIT - storm (
🥉 25 ·⭐ 3.9K ·💀 ) - Manage your SSH like a boss.MIT - fabtools (
🥉 25 ·⭐ 1.3K ·💀 ) - Tools for writing awesome Fabric files.BSD-2 - parallel-ssh (
🥉 25 ·⭐ 1K) - Asynchronous parallel SSH client library.❗️LGPL-2.1 - wssh (
🥉 17 ·⭐ 1.3K ·💀 ) - SSH to WebSockets Bridge.MIT
Process Utilities
pexpect (🥇 36 · ⭐ 2.2K) - A Python module for controlling interactive programs in a pseudo-terminal. ISC
supervisor (🥈 35 · ⭐ 7.4K) - Supervisor process control system for UNIX. ❗️Repoze Public License
ptyprocess (🥉 26 · ⭐ 180 · 💤 ) - Run a subprocess in a pseudo terminal. ISC
Asynchronous Programming
anyio (🥈 29 · ⭐ 960) - High level asynchronous concurrency and networking framework that works on.. MIT
Show 1 hidden projects...
Configuration
python-dotenv (🥇 36 · ⭐ 5.2K) - Reads key-value pairs from a .env file and can set them as.. BSD-3
omegaconf (🥈 32 · ⭐ 1.2K) - Flexible Python configuration system. The last one you will ever need. BSD-3
python-decouple (🥉 31 · ⭐ 2.2K) - Strict separation of config from code. MIT
gin-config (🥉 27 · ⭐ 1.6K) - Gin provides a lightweight configuration framework for Python. Apache-2
Show 2 hidden projects...
CLI Development
rich (🥇 42 · ⭐ 39K) - Rich is a Python library for rich text and beautiful formatting in the terminal. MIT
python-fire (🥈 36 · ⭐ 23K) - Python Fire is a library for automatically generating command.. Apache-2
python-prompt-toolkit (🥈 36 · ⭐ 7.9K) - Library for building powerful interactive command.. BSD-3
argcomplete (🥉 32 · ⭐ 1.1K · 💤 ) - Python and tab completion, better together. Apache-2
asciimatics (🥉 29 · ⭐ 3.1K) - A cross platform package to do curses-like operations, plus.. Apache-2
wcwidth (🥉 29 · ⭐ 290 · 💤 ) - Python library that measures the width of unicode strings rendered to.. MIT
ConfigArgParse (🥉 28 · ⭐ 610 · 💤 ) - A drop-in replacement for argparse that allows options to.. MIT
questionary (🥉 27 · ⭐ 920) - Python library to build pretty command line user prompts Easy to use.. MIT
Show 5 hidden projects...
- docopt (
🥈 35 ·⭐ 7.7K ·💀 ) - Pythonic command line arguments parser, that will make you smile.MIT - blessings (
🥉 29 ·⭐ 1.3K ·💀 ) - A thin, practical wrapper around terminal capabilities in Python.MIT - clint (
🥉 25 ·⭐ 78 ·💀 ) - Python Command-line Application Tools.ISC - bashplotlib (
🥉 21 ·⭐ 1.7K ·💀 ) - plotting in the terminal.MIT - docopt-ng (
🥉 21 ·⭐ 94) - Humane command line arguments parser. Now with maintenance, typehints,..MIT
Development Tools
Data Caching
cachetools (🥇 32 · ⭐ 1.4K · 📉 ) - Extensible memoizing collections and decorators. MIT
Show 1 hidden projects...
- cached-property (
🥈 30 ·⭐ 650 ·💀 ) - A decorator for caching properties in classes.BSD-3
GUI Development
kivy (🥇 40 · ⭐ 15K) - Open source UI framework written in Python, running on Windows, Linux, macOS,.. MIT
PySimpleGUI (🥈 38 · ⭐ 10K) - PySimpleGUI is a Python package that enables Python.. ❗️LGPL-3.0
DearPyGui (🥈 31 · ⭐ 8.5K) - Dear PyGui: A fast and powerful Graphical User Interface Toolkit for.. MIT
Eel (🥈 30 · ⭐ 5.1K · 💤 ) - A little Python library for making simple Electron-like HTML/JS GUI apps. MIT
Gooey (🥉 29 · ⭐ 17K) - Turn (almost) any Python command line program into a full GUI application.. MIT
Show 2 hidden projects...
- Phoenix (
🥉 28 ·⭐ 1.8K) - wxPythons Project Phoenix. A new implementation of wxPython,..❗️wxWindows - enaml (
🥉 27 ·⭐ 1.2K) - Declarative User Interfaces for Python.❗Unlicensed
Computer & Machine Vision
Machine Learning & Data Engineering
Text Data
chardet (🥇 35 · ⭐ 1.8K) - Python character encoding detector. ❗️LGPL-2.1
-
GitHub (
👨💻 48 ·🔀 240 ·📋 130 - 40% open ·⏱️ 22.07.2022):git clone https://github.com/chardet/chardet -
PyPi (
📥 58M / month ·📦 39K ·⏱️ 25.06.2022):pip install chardet -
Conda (
📥 17M ·⏱️ 25.06.2022):conda install -c conda-forge chardet -
npm (
📥 15 / month ·📦 1 ·⏱️ 20.08.2017):npm install @pypi/chardet
phonenumbers (🥈 34 · ⭐ 3.1K) - Python port of Googles libphonenumber. Apache-2
python-slugify (🥈 30 · ⭐ 1.2K) - Returns unicode slugs. MIT
inflect (🥉 29 · ⭐ 680) - Correctly generate plurals, ordinals, indefinite articles; convert numbers.. MIT
pyahocorasick (🥉 26 · ⭐ 740) - Python module (C extension and plain python) implementing Aho-.. BSD-3
Show 4 hidden projects...
- awesome-slugify (
🥉 22 ·⭐ 470 ·💀 ) - Python flexible slugify function.❗️GPL-3.0 - price-parser (
🥉 20 ·⭐ 240 ·💀 ) - Extract price amount and currency symbol from a raw text..BSD-3 - coolname (
🥉 19 ·⭐ 94 ·💀 ) - Random Name and Slug Generator.BSD-2 - millify (
🥉 14 ·⭐ 67 ·💀 ) - Convert long numbers into a human-readable format in Python.MIT
Web Development
Database Clients
Libraries for connecting to, operating, and querying databases.
SQLAlchemy (🥇 45 · ⭐ 5.8K) - The Database Toolkit for Python. MIT
azure-storage-blob (🥇 42 · ⭐ 3K) - This repository is for active development of the Azure SDK.. MIT
google-cloud-storage (🥇 40 · ⭐ 4K) - Google Cloud Client Library for Python. Apache-2
elasticsearch (🥇 40 · ⭐ 3.7K) - Official Elasticsearch client library for Python. Apache-2
MongoEngine (🥈 39 · ⭐ 3.8K) - A Python Object-Document-Mapper for working with MongoDB. MIT
kafka-python (🥈 38 · ⭐ 4.9K) - Python client for Apache Kafka. Apache-2
confluent-kafka-python (🥈 36 · ⭐ 2.9K) - Confluents Kafka Python Client. Apache-2
SQLAlchemy-Utils (🥈 36 · ⭐ 960) - Various utility functions and datatypes for SQLAlchemy. BSD-3
python-bigquery (🥈 36 · ⭐ 480) - Google BigQuery API client library. Apache-2
libcloud (🥈 35 · ⭐ 1.9K) - Apache Libcloud is a Python library which hides differences between.. Apache-2
Elasticsearch DSL (🥈 34 · ⭐ 3.5K) - High level Python client for Elasticsearch. Apache-2
AWS Data Wrangler (🥈 33 · ⭐ 3K) - Pandas on AWS - Easy integration with Athena, Glue,.. Apache-2
mysqlclient (🥈 33 · ⭐ 2.1K) - MySQL database connector for Python (with Python 3 support). ❗️GPL-2.0
PyPika (🥈 33 · ⭐ 1.7K) - PyPika is a python SQL query builder that exposes the full richness.. Apache-2
pandas-gbq (🥈 33 · ⭐ 310) - Google BigQuery connector for pandas. BSD-3
s3transfer (🥈 33 · ⭐ 150) - Amazon S3 Transfer Manager for Python. Apache-2
Prometheus Client (🥉 32 · ⭐ 2.9K) - Prometheus instrumentation library for Python.. Apache-2
Cassandra Driver (🥉 32 · ⭐ 1.3K) - DataStax Python Driver for Apache Cassandra. Apache-2
cx-Oracle (🥉 32 · ⭐ 830) - Python interface to Oracle Database now superseded by python-oracledb. BSD-3
tortoise-orm (🥉 31 · ⭐ 3K) - Familiar asyncio ORM for python, built with relations in mind. Apache-2
neo4j-driver (🥉 31 · ⭐ 730) - Neo4j Bolt driver for Python. Apache-2
dataset (🥉 29 · ⭐ 4.2K) - Easy-to-use data handling for SQL data stores with support for implicit.. MIT
py2neo (🥉 29 · ⭐ 1.1K · 💤 ) - Py2neo is a comprehensive Neo4j driver library and toolkit for.. Apache-2
redis-py-cluster (🥉 29 · ⭐ 1.1K) - Python cluster client for the official redis cluster. Redis.. MIT
sqlmodel (🥉 25 · ⭐ 7.9K) - SQL databases in Python, designed for simplicity, compatibility,.. MIT pydantic
prisma (🥉 23 · ⭐ 760) - Prisma Client Python is an auto-generated and fully type-safe database.. Apache-2
ODMantic (🥉 21 · ⭐ 610) - Async ODM (Object Document Mapper) for MongoDB based on python type hints. ISC
aioprometheus (🥉 20 · ⭐ 120) - A Prometheus Python client library for asyncio-based applications. MIT
psycopg3 (🥉 18 · ⭐ 760) - New generation PostgreSQL database adapter for the Python.. ❗️LGPL-3.0
-
GitHub (
👨💻 26 ·🔀 66 ·📋 180 - 12% open ·⏱️ 17.08.2022):git clone https://github.com/psycopg/psycopg
Show 11 hidden projects...
- psycopg2 (
🥈 37 ·⭐ 2.7K) - PostgreSQL database adapter for the Python..❗️BSD-3-Clause-Attribution - pyodbc (
🥉 32 ·⭐ 2.5K) - Python ODBC bridge.❗️MIT-0 - google-cloud-bigtable (
🥉 30 ·⭐ 34) - Google Cloud Bigtable API client library.Apache-2 - Records (
🥉 29 ·⭐ 6.9K ·💀 ) - SQL for Humans.ISC - mongo-connector (
🥉 26 ·⭐ 1.9K ·💀 ) - MongoDB data stream pipeline tools by YouGov (adopted..Apache-2 - pyhdb (
🥉 24 ·⭐ 300 ·💀 ) - SAP HANA Connector in pure Python.Apache-2 - PyMODM (
🥉 22 ·⭐ 340 ·💀 ) - A Pythonic, object-oriented interface for working with MongoDB.Apache-2 - db.py (
🥉 21 ·⭐ 1.2K ·💀 ) - db.py is an easier way to interact with your databases.BSD-2 - gsheets-db-api (
🥉 19 ·⭐ 180 ·💀 ) - A Python DB-API and SQLAlchemy dialect to Google Spreasheets.MIT - SuperSQLite (
🥉 15 ·⭐ 700 ·💀 ) - A supercharged SQLite library for Python.MIT - lazydata (
🥉 15 ·⭐ 630 ·💀 ) - Lazydata: Scalable data dependencies for Python projects.Apache-2
Data Loading & Extraction
Libraries for loading, collecting, and extracting data from a variety of data sources and formats.
Datasets (🥇 42 · ⭐ 14K) - The largest hub of ready-to-use datasets for ML models with fast,.. Apache-2
xmltodict (🥈 35 · ⭐ 4.9K) - Python module that makes working with XML feel like you are working.. MIT
python-magic (🥈 35 · ⭐ 2.2K) - A python wrapper for libmagic. MIT
snorkel (🥈 32 · ⭐ 5.2K) - A system for quickly generating training data with weak supervision. Apache-2
csvkit (🥈 32 · ⭐ 5.1K) - A suite of utilities for converting to and working with CSV, the king of.. MIT
smart-open (🥈 31 · ⭐ 2.6K) - Utils for streaming large files (S3, HDFS, gzip, bz2...). MIT
gdown (🥈 30 · ⭐ 2.3K) - Download a large file from Google Drive (curl/wget fails because of the.. MIT
pandas-datareader (🥈 29 · ⭐ 2.4K) - Extract data from a wide range of Internet sources into a.. BSD-3
Intake (🥉 27 · ⭐ 790) - Intake is a lightweight package for finding, investigating, loading and.. BSD-2
rows (🥉 23 · ⭐ 810) - A common, beautiful interface to tabular data, no matter the format. ❗️LGPL-3.0
img2dataset (🥉 22 · ⭐ 860) - Easily turn large sets of image urls to an image dataset. Can.. MIT
deepdish (🥉 22 · ⭐ 240 · 💤 ) - Flexible HDF5 saving/loading and other data science tools from the.. BSD-3
csvs-to-sqlite (🥉 16 · ⭐ 710 · 💤 ) - Convert CSV files into a SQLite database. Apache-2
Show 9 hidden projects...
- PDFMiner (
🥈 29 ·⭐ 4.9K ·💀 ) - Python PDF Parser (Not actively maintained). Check out pdfminer.six.MIT - tabulator-py (
🥉 27 ·⭐ 230 ·💀 ) - Python library for reading and writing tabular data via streams.MIT - messytables (
🥉 25 ·⭐ 380 ·💀 ) - Tools for parsing messy tabular data. This is now superseded by..MIT - pyexcel-xlsx (
🥉 24 ·⭐ 110 ·💀 ) - A wrapper library to read, manipulate and write data in xlsx..BSD-3 - camelot (
🥉 23 ·⭐ 1.6K ·💀 ) - A Python library to extract tabular data from PDFs.MIT - Singer (
🥉 23 ·⭐ 1K ·💀 ) - Standard for moving data between databases, web APIs, files,..❗️AGPL-3.0 - borb (
🥉 19 ·⭐ 2.7K) - borb is a library for reading, creating and manipulating PDF files..❗Unlicensed - excalibur (
🥉 19 ·⭐ 1.2K ·💀 ) - A web interface to extract tabular data from PDFs.MIT - Upgini (
🥉 17 ·⭐ 83) - Free automated data enrichment library for machine learning searches..BSD-3
Data Pipelines & Streaming
Libraries for data batch- and stream-processing, workflow automation, job scheduling, and other data pipeline tasks.
Airflow (🥇 45 · ⭐ 27K) - Platform to programmatically author, schedule, and monitor workflows. Apache-2
-
GitHub (
👨💻 2.5K ·🔀 11K ·📥 330K ·📋 6.1K - 13% open ·⏱️ 18.08.2022):git clone https://github.com/apache/airflow -
PyPi (
📥 9M / month ·📦 480 ·⏱️ 06.07.2022):pip install apache-airflow -
Conda (
📥 690K ·⏱️ 27.07.2022):conda install -c conda-forge airflow -
Docker Hub (
📥 82M ·⭐ 380 ·⏱️ 09.07.2022):docker pull apache/airflow
Celery (🥇 45 · ⭐ 20K) - Asynchronous task queue/job queue based on distributed message passing. BSD-3
luigi (🥈 38 · ⭐ 16K · 📈 ) - Luigi is a Python module that helps you build complex pipelines of.. Apache-2
Great Expectations (🥈 37 · ⭐ 7K) - Always know what to expect from your data. Apache-2
Kedro (🥈 35 · ⭐ 7.5K) - A Python framework for creating reproducible, maintainable and modular.. Apache-2
dbt (🥈 35 · ⭐ 5.3K) - dbt enables data analysts and engineers to transform their data using the.. Apache-2
Activeloop (🥈 31 · ⭐ 4.7K) - Dataset format for AI. Build, manage, query & visualize datasets.. MPL-2.0
whylogs (🥉 27 · ⭐ 1.7K) - Open standard for end-to-end data and ML monitoring for any scale in.. Apache-2
dbnd (🥉 27 · ⭐ 230) - DBND is an agile pipeline framework that helps data engineering teams.. Apache-2
PyFunctional (🥉 26 · ⭐ 2.1K) - Python library for creating data pipelines with chain functional.. MIT
Optimus (🥉 26 · ⭐ 1.2K) - Agile Data Preparation Workflows madeeasy with Pandas, Dask,.. Apache-2 spark
streamparse (🥉 25 · ⭐ 1.5K) - Run Python in Apache Storm topologies. Pythonic API, CLI.. Apache-2
BatchFlow (🥉 21 · ⭐ 180) - BatchFlow helps you conveniently work with random or sequential.. Apache-2
bodywork-core (🥉 20 · ⭐ 400) - ML pipeline orchestration and model deployments on.. ❗️AGPL-3.0
-
GitHub (