Skip to content
Thibault Bétrémieux edited this page Oct 3, 2023 · 10 revisions

Welcome to the pangres wiki!

Quickstart

Install pangres with:

pip install pangres

Here is a basic usage example (see more in the sections below):

import pandas as pd
from pangres import upsert
from sqlalchemy import create_engine

# create or get a pandas DataFrame
data = {'id': [10],
        'name': ['Albert']}
df = pd.DataFrame(data)
df = df.set_index('id')  # a unique index is required!

# get a SQL engine from sqlalchemy (used to connect to the SQL database)
engine = create_engine('sqlite://')
table_name = 'test'

# update given SQL table using the DataFrame
upsert(
    con=engine,
    df=df,
    table_name=table_name,
    if_row_exists='update',
    dtype=None,  # same logic as the parameter in pandas.to_sql
    chunksize=1000,
    create_table=True  # create a new table if it does not exist
)

Usage

Upserting DataFrames in SQL

Upserting DataFrames in SQL asynchronously

Checking and adjusting the size of chunks to upsert

Fix bad column names for Postgres

Logging

See also demo notebooks below.

Demo notebooks

Demo notebook for pangres.upsert

Demo notebook for pangres.upsert with a progress bar

Notebook with transaction control and commit-as-you-go workflows example

Gotchas with asynchronous upserts (pangres.aupsert)

Notes

Parts of the documentation were automatically generated using the docstrings of pangres' functions/classes/methods via my library npdoc_to_md.