Schema Generation

Schema Generation

Use generate_from_schema when you already know the expected table structure and need a Pandas or Spark DataFrame with synthetic values.

from great_generator import generate_from_schema

schema = "customer_id int, customer_name string, email string"
df = generate_from_schema(schema, rows=1000)

Required values

Use required_values when your query, dashboard, ETL logic, or test case expects specific values.

df = generate_from_schema(
    schema=schema,
    rows=100_000,
    required_values={
        "region": ["SOUTH"],
        "status": ["ACTIVE"],
    },
    seed=42,
)

The generated data will include those values. Other values may also appear unless you explicitly configure the generator otherwise.

Partition-aware generation

Use partition_by when you want generated data for specific partition dates or partition values.

df = generate_from_schema(
    schema=schema,
    rows=300_000,
    partition_by={
        "column": "business_date",
        "values": ["2026-01-01", "2026-01-02", "2026-01-03"],
        "distribution": "balanced",
    },
    seed=42,
)

With balanced distribution, Great Generator creates equal or near-equal counts across the listed partition values.

Target selectivity

Use target_selectivity when you want approximate control over how often required values appear.

df = generate_from_schema(
    schema=schema,
    rows=100_000,
    required_values={
        "region": ["SOUTH"],
    },
    target_selectivity={
        "region": {
            "SOUTH": 0.25,
        },
    },
    seed=42,
)

This asks the generator to make about 25% of rows contain region = "SOUTH".

Selectivity is approximate unless exact-count mode is explicitly supported. If the target cannot be matched exactly, the coverage report shows the actual result and any warnings.

Default behavior

All query-aware options are optional. Existing generation behavior is unchanged unless you provide required_values, partition_by, target_selectivity, ensure_join_coverage, or query_profile.