DATAMIMIC CE v3.0.0

[ake2l]

Time-series generation is now a first-class DSL primitive. Define start, end, and interval on <generate>, and the engine produces the temporal sequence directly. No external orchestration, no Python wrapper, no count math.

This release also closes the remaining gaps in seeded coverage (null injection, pattern, weighted distribution, nested cardinality, source shuffling) and contains one breaking change in DateFormatConverter. Read the Migration Notes.

---

✨ Time-Series Generation

Generate temporal sequences from the DSL. Count is derived from the window and interval. Three names are available in the script namespace: ts.now (current datetime), ts.step (0-based index), ts.series (full series list). Do not name a key ts — it shadows the namespace.

Stock ticks at 1-minute resolution:

<setup rngSeed="1">
  <generate name="ticks"
            start="2024-01-15T09:00:00"
            end="2024-01-15T17:00:00"
            interval="PT1M"
            target="CSV">
    <key name="timestamp" script="ts.now" />
    <key name="price"     script="round(100 + ts.step * 0.05, 2)" />
    <key name="volume"    type="int" />
  </generate>
</setup>

Daily sensor readings with a seasonal curve:

<setup rngSeed="3">
  <generate name="sensor_data"
            start="2024-01-01"
            end="2024-12-31"
            interval="P1D"
            target="CSV">
    <key name="date"        script="ts.now" />
    <key name="day_of_year" script="ts.step + 1" />
    <key name="temperature" script="round(15 + 10 * math.sin(ts.step / 58.0), 1)" />
  </generate>
</setup>

Sub-second log stream:

<setup rngSeed="5">
  <generate name="logs"
            start="2024-06-01T00:00:00"
            end="2024-06-01T01:00:00"
            interval="PT0.5S"
            target="JSON">
    <key name="timestamp" script="ts.now" />
    <key name="level"     values="('INFO','INFO','INFO','WARN','ERROR')" />
    <key name="message"   pattern="[A-Z]{4}-[0-9]{4}" />
  </generate>
</setup>

What you can build with this: IoT telemetry, log replay, market-tick streams, scheduled-event histories, time-bucketed audit data. Combined with a seed, the entire temporal sequence is reproducible.

---

✨ Model-Wide Seed Control

Set rngSeed on <setup> to lock an entire model. Same seed, same output. Omit it and behaviour is unchanged.

<setup rngSeed="42">
  <generate name="customers" count="1000" target="CSV">
    <variable name="person" entity="Person" />
    <key name="id"    type="int" />
    <key name="name"  script="person.full_name" />
    <key name="email" script="person.email" />
  </generate>
</setup>

A per-variable rngSeed forks an independent stream without affecting siblings:

<setup rngSeed="42">
  <generate name="products" count="500" target="CSV">
    <variable name="product" entity="Product" rngSeed="99" />
    <key name="id"    type="int" />
    <key name="name"  script="product.name" />
    <key name="price" script="product.price" />
  </generate>
</setup>

---

Improvements

Seeded coverage extended to the remaining DSL constructs. nullQuota, values, pattern, nestedKey cardinality, weighted .wgt.csv sampling, and shuffled source reads (CSV, .ent.csv, JSON, SQLite, cascading nested <generate>) now derive from the same <setup rngSeed>. No DSL changes required. Existing seeded projects gain reproducibility on these constructs with no change to record streams that were already deterministic.

Example — seeded nullQuota, values, pattern, and nestedKey:

<setup rngSeed="42">
  <generate name="orders" count="20" target="JSON">
    <key name="order_id"  type="int" />
    <key name="reference" pattern="REF-[0-9]{6}-[A-Z]{3}" />
    <key name="status"    values="('active','inactive','pending')" />
    <key name="comment"   type="string" nullQuota="0.3" />
    <nestedKey name="items" type="list" minCount="1" maxCount="5">
      <key name="sku"      pattern="SKU-[0-9]{4}" />
      <key name="quantity" type="int" />
    </nestedKey>
  </generate>
</setup>

---

Migration Notes

DateFormatConverter now returns str. Previously it returned a datetime. If you chain DateFormatConverter into a converter that expects datetime, update the chain to consume str. This change also fixes the Feb 29 crash that occurred when format strings without %Y defaulted to year 1900.

generate_random_value_based_on_type requires rng as a keyword argument. Direct Python call sites must pass rng=. DSL users are not affected.

DSL backward compatibility. New attributes (rngSeed, start, end, interval) are additive. Existing 2.x XML models parse and run unchanged.

---

Fixes

• test_medical_procedure_flags flake. 30% surgical probability over 10 records produced a ~2.8% per-run failure rate. Now seeded.
• PoliceOfficer.birthdate return type annotation corrected from str to datetime.

---

This discussion was created from the release DATAMIMIC CE v3.0.0.