v0.8.1

This is a patch release to fix a packaging issue where missing optional dependencies could cause an ImportError on startup.

Bug Fixes

• Core Imports: Updated __init__.py to use resilient try/except blocks for all external connectors (Kafka, PyArrow, etc.). Users can now run core modules or the new storage tools (e.g., pip install dynamic-des[parquet]) without encountering startup crashes from missing aiokafka dependencies.

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Summary

This release transforms Dynamic DES from a real-time monitoring tool into a full-lifecycle simulation platform. v0.8.0 introduces high-performance historical data generation and native Data Lake egress, allowing users to fast-forward through years of simulation time or transform their live models into synchronized forecasting engines. By exporting schema-enforced Parquet or JSONL files natively to multi-cloud object storage, this release enables a seamless transition from live digital twin monitoring to massive synthetic dataset generation for Machine Learning and analytical backfilling.

🚀 Key Features & Improvements

• Historical Fast-Forwarding & Forecasting: Introduced support for Time Dilation using factor=0.0. Simulations now run as fast as your CPU allows, detaching logical simulation time from real-world wall-clock time for rapid data generation and future-state prediction.
• Native Data Lake Egress (PyArrow VFS): Added ParquetStorageEgress and JsonlStorageEgress connectors. Powered by PyArrow, these support automatic file rotation (UUID chunking), heavy compression, and lock-free writes directly to Local, AWS S3, Google Cloud Storage (GCS), Azure Blob, or SeaweedFS.
• Schema Drift Protection: Implemented strict schema inference and enforcement for columnar Parquet storage. The engine now caches the initial batch schema and automatically casts subsequent records to prevent pipeline-breaking data type changes.
• Dynamic Payload Routing (Multiplexing): Storage connectors now support a custom path_router callable. This allows users to dynamically split events, separate telemetry, or drop meaningless metrics on the fly before they are written to disk.
• Unified Event Buffering: Telemetry metrics now share the primary event buffer. This ensures that low-volume metrics are batched alongside lifecycle events, preventing the "tiny file problem" in storage buckets and reducing network overhead for Kafka.
• Resilient Teardown Mechanism: Upgraded the environment shutdown process with a 5-second queue-draining safety net. This guarantees that final events are flushed to disk or network before exit, preventing data loss without risking system deadlocks.
• Operational Observability: Introduced real-time heartbeat logging for background storage writers. Users can now monitor batch progress and queue depth during long-running historical batch runs.
• New Forecasting CLI Example: Added the ddes-history-example command to demonstrate a full week of factory simulation multiplexed and exported directly to chunked Parquet files in seconds.

Full Changelog: v0.7.0...v0.8.0

v0.7.0

Summary

This release features a massive overhaul of the documentation and developer experience, introducing zero-setup CLI demos and comprehensive guides for enterprise integrations. The documentation now fully covers advanced use cases, including pluggable Avro serialization, strict Pydantic duck-typing, and secure Kafka cluster authentication.

🚀 Key Features & Improvements

• Zero-Setup CLI Demos: Introduced built-in CLI commands (ddes-local-example, ddes-kafka-infra-up, ddes-kafka-example, and ddes-kafka-dashboard) allowing users to launch dependency-free local simulations or full Kafka-backed digital twins directly from the terminal.
• Advanced Serialization Guide: Published a detailed walkthrough on using Pydantic duck-typing and configuring Pluggable Topic Routing to seamlessly mix JsonSerializer and ConfluentAvroSerializer streams on the same connection.
• Enterprise Security Documentation: Added comprehensive examples for connecting Dynamic DES to secure environments, including Confluent Cloud (SASL PLAIN), strictly authenticated on-premise clusters (mTLS), and AWS MSK (IAM / OAUTHBEARER).
• API Reference Completion: Updated the api.md reference to explicitly expose all native Serializers and Deserializers (JSON, Confluent Avro, and AWS Glue Avro).
• Core Concepts Deep-Dive: Expanded the foundational documentation to thoroughly explain the "Switchboard Pattern," asynchronous network decoupling, and the dynamic capacity shrinking paradox.

Full Changelog: v0.6.0...v0.7.0

v0.6.0

Summary

This release enhances the SimulationRegistry to fully support dictionary-based configurations, ensuring that custom user-defined variables are correctly registered and synchronized during runtime updates.

🚀 Key Features & Improvements

• Dictionary State Synchronization: Updated the SimulationRegistry to intelligently differentiate between dataclasses and raw dictionaries. The registry now safely uses item assignment (dict["key"] = value) to synchronize custom variables, preventing attribute errors.
• Dynamic Variable Registration: The registry now natively iterates through the variables dictionary upon initialization, automatically wrapping each custom state in a DynamicValue. This allows external systems to update custom variables on the fly and trigger SimPy events just like standard parameters.

Full Changelog: v0.5.0...v0.6.0

v0.5.0

Summary

This release introduces the lag_monitor_interval parameter to EgressMixIn, providing developers with native control over how frequently system lag is measured and tracked during simulations.

🚀 Key Features

• Configurable Lag Monitoring: Customize the lag_monitor_interval to define exactly how often lag telemetry is published. By default, telemetry is emitted every 1 second.
• Flexible Telemetry Control: Easily adjust the monitoring frequency to suit your specific tracking needs, or set the interval to None to completely disable lag event publishing.

Full Changelog: v0.4.0...v0.5.0

v0.4.0

Summary

This release expands the SimParameter schema by introducing a variables attribute, giving developers a native way to manage custom state data and control logic directly through the simulation registry.

🚀 Key Features

• Custom State Tracking: A flexible variables: Dict[str, Any] field designed to hold arbitrary user-defined data (e.g., int, float, bool, str) for custom machine states or logic flags.
• External Control Dials: These variables are fully compatible with the SimulationRegistry, allowing you to update custom parameters on the fly without needing to write custom wrapper classes.

Full Changelog: v0.3.0...v0.4.0

v0.3.0

Summary

This release introduces multiplexing to the KafkaEgress connector, allowing developers to define custom routing logic for outbound simulation data while maintaining 100% backward compatibility.

🚀 Key Features

• Custom topic_router: You can now pass a Callable to KafkaEgress to dynamically determine the destination Kafka topic on a per-message basis.
  • Multiplexing: Easily split complex data streams (e.g., routing prediction_request events to a dedicated ML topic while keeping standard events on the main lifecycle topic).
• Zero Breaking Changes: Existing deployments using the default telemetry_topic and event_topic routing will continue to function exactly as before without any code updates.

Example

def custom_topic_router(data: dict) -> str:
    stream_type = data.get("stream_type")
    if stream_type == "telemetry":
        return "sim-telemetry"

    value = data.get("value", {})
    if isinstance(value, dict):
        event_type = value.get("event_type")
        if event_type == "prediction_request":
            return "mill-predictions"
        elif event_type == "ground_truth":
            return "mill-groundtruth"

    return "mill-lifecycle"

# Pass it to the egress connector
egress = KafkaEgress(
    bootstrap_servers="localhost:9092",
    topic_router=custom_topic_router
)

Full Changelog: v0.2.0...v0.3.0

v0.2.0

Summary

This release adds DynamicContainer and DynamicStore, enabling real-time capacity updates for continuous materials and discrete item buffers via the SimulationRegistry.

Key Features

• DynamicContainer: Built for continuous flow modeling (fluids, gases, bulk inventory) with support for fractional (float) capacity and level updates.
• DynamicStore: Designed for discrete item buffers (warehouses, staging areas). Supports an optional priority=True flag to enable simpy.PriorityStore behavior.
• Runtime Control: Both objects actively listen for registry updates to adjust capacities and re-evaluate pending put requests instantly.

---

Full Changelog: v0.1.0...v0.2.0

v0.1.1

v0.1.1 - PyPI Rendering Patch

This is a quick patch release to fix a documentation rendering issue on PyPI.

Bug Fixes

• Documentation: Updated the README.md to use absolute GitHub raw URLs for image assets. This ensures the dashboard preview GIF renders correctly on the official PyPI package page instead of showing a broken link.

Full Changelog: v0.1.0...v0.1.1

v0.1.0

Dynamic DES is a high-performance, real-time control plane for SimPy.

It bridges the gap between static discrete-event simulations and the live world by allowing you to update simulation parameters (arrivals, service times, capacities) and stream telemetry and events via Kafka, Redis, or PostgreSQL without stopping the simulation.

Key Features

• ⚡ Real-Time Control: Synchronize SimPy with the system clock using DynamicRealtimeEnvironment.
• 🔗 Dynamic Registry: Dynamic, path-based updates (e.g., Line_A.arrival.rate) that trigger instant logic changes.
• 🚀 High Throughput: Optimized to handle high throughput using orjson and local batching.
• 🔋 Flexible Resources: DynamicResource provides prioritized queuing with graceful capacity shrinking.
• 🔌 Modular Connectors: Plugin-based architecture for Kafka, Redis, Postgres and Local testing.
• 📊 System Observability: Built-in lag monitoring to track simulation drift from real-world time, exposed via the telemetry stream.

📚 Quick Start

Install the engine with Kafka and Dashboard support:

pip install dynamic-des[kafka,dashboard]

Run the full Real-Time Digital Twin stack:

# Start Kafka cluster (requires Docker)
ddes-kafka-infra-up

# Run the simulation (new terminal)
# Ctrl + C to stop
ddes-kafka-example

# Start the dashboard (new terminal)
# Opens in browser at http://localhost:8080
# Ctrl + C to stop
ddes-kafka-dashboard

# Clean up the infrastructure
ddes-kafka-infra-down

Check the documentation for setup guides, API references, and architecture details.