DATAMIMIC CE 4.2.0

DATAMIMIC CE 4.2.0 makes seeded runs reproducible on the same runtime across all generators, entities and script expressions, turns the Hash converter into a keyed hash, and gives database paging a stable order.

Highlights

• One seed for the whole model. With <setup rngSeed="N"> every generator, entity and script expression replays on the same Python and machine, including random, uuid, fake and the current date and time (fixed at 2025-01-01 12:00). UUIDGenerator, PasswordGenerator, TokenGenerator and HashGenerator now follow the seed like every other generator. CI replays every generator and script path in two separate processes. (#219, #240)
• Keyed Hash converter. Hash(...) is now an HMAC. <setup rngSeed> is the key, so the same value gives the same token in every run; without a seed each run gets a new random key, shared by all workers of the run. (#226, #247)
• Stable database paging. Every paged RDBMS read is ordered by the primary key, or by all columns. A selector's own ORDER BY stays first. Selectors and <execute> SQL scripts are read with sqlglot in the database's dialect, so semicolons in literals and comments, Oracle PL/SQL blocks and SQL Server ORDER BY selectors work. (#228, #238, #241–#244)
• Timezone-independent timestamps. Date2Timestamp, Timestamp2Date and epoch date formats give the same result on every host timezone. (#220)
• Explicit text encoding. CE no longer depends on the platform's default text encoding. UTF-8 is the default; explicitly configured encodings such as cp1252 stay supported. (#221)
• XML sources. <generate source="x.xml"> without count reads every <item>. (#222)
• Authoring. verified is blocked when a product of an exporting model has no target, and database/MongoDB clients are reported as unsupported_intent. (#223, #225)

Behaviour changes

The same model can produce different data than 4.1.0:

• Hash converter tokens change in every mode. Regenerate stored tokens or pseudonyms once.
• Seeded runs: values from UUIDGenerator, PasswordGenerator, TokenGenerator, HashGenerator and from random, uuid, fake and the current time in script expressions change, and are now reproducible.
• Database sources: paged reads and full table reads are now ordered by the primary key or all columns, so row order can differ from 4.1.0; the rows themselves do not.
• Timestamps on hosts outside UTC.
• Authoring can report verified=false where 4.1.0 reported true.

Two cases that ran before now stop with an error:

• A seeded script expression calling np.random, random.SystemRandom, os.urandom or uuid.uuid1. Use random.* or a generator instead.
• A database table without a primary key, or a selector, that returns a column type the database cannot sort (PostgreSQL json, Oracle CLOB, SQL Server text/ntext/image/xml). Add a primary key, or select or cast those columns.

Upgrade notes

• New dependency: sqlglot (>=30.18,<31).
• SetupContext(...) takes run_seed= instead of seed= and current_seed= (internal API).

Verification

Each change ships with regression coverage that fails on 4.1.0, including cross-process replay of every generator and script path, keyed Hash across workers, and database paging on PostgreSQL, SQL Server and Oracle. Seeded output of existing distribution models is byte-identical across the seed refactor.