Roadmap

Roadmap

The roadmap keeps schema-based lower-environment generation at the center.

Schema ingestion

• rich inline schema metadata
• full SQL CREATE TABLE parsing with dialect adapters
• JSON Schema and schema files
• YAML schema profiles
• column-list inference
• SQLAlchemy models
• Pydantic models
• Python dataclasses
• richer nested object, array, map, and struct support

Generation and scale

• Spark-native arbitrary-schema generation
• chunked Pandas generation
• explicit reference date for time-stable reproducibility
• generation manifests with schema, rules, counts, and lineage metadata

Data quality

• richer quality profiling
• relationship validation reports
• configurable uniqueness and regex constraints
• named edge-case scenarios
• anomaly labels for Spark outputs

Platform and community

• more tested storage recipes
• additional domain packs and lifecycle state machines
• versioned recipe schema
• streaming and Kafka-oriented output adapters

Planned items are not documented as supported until their behavior and tests land in the library.