-
Notifications
You must be signed in to change notification settings - Fork 4
Supported Schema Input Types
Ravi Kiran Pagidi edited this page Jun 28, 2026
·
1 revision
This matrix describes the current code, not aspirational support.
| Schema Input Type | Example | Best For | Status |
|---|---|---|---|
| Plain Python mapping | {"name": "string", "age": "int"} |
Fast table generation | Supported |
| Rich inline metadata | {"age": {"type": "int", "min": 18}} |
Embedded business rules |
Partial: use separate custom_rules today |
| Pandas dtype mapping | df.dtypes.to_dict() |
Pandas and notebooks | Supported |
| Pandas DataFrame | empty or populated frame | Preserve Pandas dtypes | Supported |
| Compact DDL | "id int, name string" |
SQL-like definitions | Supported |
| Full SQL DDL | CREATE TABLE ... |
Database teams | Planned |
| PySpark StructType | StructType([...]) |
Spark and lakehouse platforms | Supported for common scalar fields |
| PySpark DataFrame | Spark DataFrame | Infer schema and SparkSession | Supported |
| Great Generator TableSchema | typed object | Extensions | Supported |
| Great Generator DomainSchema | multi-table metadata | Schema-based related tables | Supported |
| JSON Schema | object with properties
|
APIs and contracts | Planned |
| YAML schema profile | schema file | Reusable configuration | Planned |
| Column list | ["name", "age"] |
Fast prototypes | Planned |
| SQLAlchemy model | ORM class | Backend teams | Planned |
| Pydantic model | BaseModel | API teams | Planned |
| Dataclass | Python dataclass | Typed Python | Planned |
JSON, TOML, and simple YAML dataset recipes are supported by generate_from_recipe. That is separate from schema input support.
- Python, compact DDL, and Pandas inputs return Pandas by default.
- A Spark context or
engine="spark"returns a Spark DataFrame. - A PySpark DataFrame provides its own SparkSession.
-
DomainSchemareturns a dictionary of DataFrames.
See the individual pages for examples and limitations.