Pandas Schema Examples

Pandas Schema Examples

Status

Supported. Both dtype mappings and DataFrames can be schema sources.

What it is and when to use it

Use Pandas schemas for local development, Jupyter notebooks, analytics engineering, and tests that already define expected DataFrame dtypes.

Example

import pandas as pd
from great_generator import generate_from_schema

empty = pd.DataFrame(
    {
        "employee_id": pd.Series(dtype="string"),
        "employee_name": pd.Series(dtype="string"),
        "employee_age": pd.Series(dtype="int64"),
        "salary": pd.Series(dtype="float64"),
        "hire_date": pd.Series(dtype="datetime64[ns]"),
    }
)

from_frame = generate_from_schema(empty, rows=500, domain="hr")
from_dtypes = generate_from_schema(empty.dtypes.to_dict(), rows=500, domain="hr")
print(from_frame.head())

Sample output

employee_id  employee_name  employee_age  salary    hire_date
EMP000001    Jordan Smith   41            98215.40  2024-03-12

Write output

from_frame.to_csv("employees.csv", index=False)
from_frame.to_parquet("employees.parquet", index=False)

Notes and limitations

Input records are not sampled or copied. The DataFrame supplies column names and dtypes. Some permissive Pandas extension dtypes may remain object-like if casting is not possible.