Home

Great Generator Wiki

Great Generator helps engineers generate realistic synthetic data from schema definitions for lower environments, data pipelines, testing, demos, and analytics projects.

Start here if you have a schema but do not have safe test data.

from great_generator import generate_from_schema, parse_ddl

schema = {
    "customer_id": "string",
    "customer_name": "string",
    "age": "int",
    "email": "string",
    "balance": "float",
    "created_at": "datetime",
}

df = generate_from_schema(schema, rows=1000)

What it solves

Teams often cannot copy production records into dev, QA, SIT, UAT, sandbox, demo, or performance environments. Great Generator creates synthetic, non-production data from your schema so you can test without waiting for a production extract.

Start here

1. Problem Statement
2. Quick Start with generate_from_schema
3. Generate Related Tables
4. Supported Schema Input Types
5. Realistic Mode
6. Custom Rules
7. Writing Data to Files and Databases
8. Data Quality and Edge Cases
9. Query-Aware Generation
10. AI advisor layer

Schema-first or domain-first?

• Use generate_from_schema for real project structures and lower-environment testing.
• Use generate_relational when several custom tables need valid key relationships.
• Query-aware generation: Generate synthetic data that includes required query filter values, partition dates, and relational join paths for safer SQL and pipeline testing.
• Use generate_domain for ready-made demonstrations, tutorials, and learning datasets.
• Use the optional advisor layer to create editable plans, column tags, and realism reviews before deterministic generation.

Important boundary

Great Generator creates synthetic data. It does not anonymize, mask, de-identify, or transform real production data. Follow your organization's data governance, privacy, security, and compliance policies.

Contract-first SQL DDL

Use parse_ddl(...) to convert SQL CREATE TABLE DDL into canonical contracts with stable hashes, keys, relationships, and structured parser diagnostics. See SQL DDL Schema Examples.

Query-aware generation

Great Generator can create synthetic data that contains the values, partition dates, and join paths needed by your SQL queries. This helps teams test SQL logic, partition pruning, joins, aggregations, and performance behavior without using production data.

All query-aware options are optional. Existing generation behavior is unchanged unless you provide required_values, partition_by, target_selectivity, ensure_join_coverage, or query_profile.

See Query-Aware Generation for examples.