# Developer Guide This guide explains how to configure Morphium and use its core APIs. ## Configuration Model - `MorphiumConfig` aggregates dedicated settings objects. Use these nested accessors: - `connectionSettings()` – database name, pool sizes, timeouts - `clusterSettings()` – host seed, replica set - `driverSettings()` – driver name (`PooledDriver`, `SingleMongoConnectDriver`, `InMemDriver`), idle sleeps - `messagingSettings()` – queue name, window size, multithreading, change streams, poll pause - `cacheSettings()` – global TTL, housekeeping, cache implementation - `threadPoolSettings()` – async operation thread pool - `writerSettings()` – write buffer behavior and writer implementation - `objectMappingSettings()` – camelCase conversion, lifecycle options - `encryptionSettings()` – value and credentials encryption providers/keys - `collectionCheckSettings()` – index/capped checks - `authSettings()` – MongoDB credentials ### Example ```java MorphiumConfig cfg = new MorphiumConfig(); cfg.connectionSettings().setDatabase("myapp"); cfg.clusterSettings().addHostToSeed("mongo1", 27017); cfg.clusterSettings().addHostToSeed("mongo2", 27017); cfg.driverSettings().setDriverName("PooledDriver"); // default // Optional: messaging defaults cfg.messagingSettings().setMessageQueueName("msg"); cfg.messagingSettings().setMessagingWindowSize(100); cfg.messagingSettings().setMessagingMultithreadded(true); cfg.messagingSettings().setUseChangeStream(true); ``` ## Object Mapping - Use annotations on POJOs: `@Entity`, `@Embedded`, `@Id`, `@Reference(lazyLoading=true)`, `@Cache`, `@Index` - Field‑level encryption: `@Encrypted` (requires an encryption provider/key) _hint_: All times and timeout settings are in milliseconds throughout whole Morphium ### Example ```java @Entity(translateCamelCase = true) @Cache(timeout = 60_000) public class Order { @Id private MorphiumId id; @Index private String customerId; private BigDecimal amount; @Reference(lazyLoading = true) private List items; @Embedded private Address shippingAddress; } ``` ## Embedded vs Reference - Embedded (`@Embedded`): - No MongoDB `_id` required; data is stored inline inside the parent document. - Ideal for value objects and nested structures (address, money, coords). - One read/write touches a single MongoDB document; deserialization splits into Java objects. - Supports `typeId`, `translateCamelCase`, and `polymorph` like `@Entity`. - Reference (`@Reference`): - Stores only the target entity’s ID in the parent; the referenced entity lives in its own collection/document. - Reading references may incur N+1 queries: one for the parent plus one per referenced entity (unless `lazyLoading=true` defers loads until first access). - Use for large/independent aggregates or when the referenced object changes on its own lifecycle. - Supports `cascadeDelete=true` to delete referenced entities when the parent is deleted, and `orphanRemoval=true` to delete referenced entities that are removed from the parent during an update. - Circular `@Reference` chains are handled by cycle detection during serialization. For bidirectional references, use `lazyLoading=true` on at least one side. - See `docs/howtos/references-and-relationships.md` for a comprehensive guide. ### Simple Example ```java @Embedded(typeId = "Address") public class Address { private String street; private String city; } @Entity(typeId = "Customer") public class Customer { @Id private MorphiumId id; private String name; } @Entity(typeId = "Order") public class Order { @Id private MorphiumId id; // Embedded: stored inline in the Order document @Embedded private Address shippingAddress; // Reference: only the Customer ID is stored in Order; Customer lives in its own collection @Reference(lazyLoading = true) private Customer customer; } ``` #### Notes - When reading an `Order`, Morphium returns `shippingAddress` from the same document. The `customer` is loaded on first access due to `lazyLoading` (otherwise N+1 reads if you eagerly access many references). - The MongoDB `orders` collection holds full address fields inline; customers are separate documents in the `customer` collection. ### Further Examples in Tests - `EmbeddedObject` used across tests: src/test/java/de/caluga/test/mongo/suite/data/EmbeddedObject.java - GitHub: https://github.com/sboesebeck/morphium/blob/develop/src/test/java/de/caluga/test/mongo/suite/data/EmbeddedObject.java - `ComplexObject` demonstrates embedded lists and references: src/test/java/de/caluga/test/mongo/suite/data/ComplexObject.java - GitHub: https://github.com/sboesebeck/morphium/blob/develop/src/test/java/de/caluga/test/mongo/suite/data/ComplexObject.java Note: these test classes are intentionally technical and cover edge cases. ## Stable Type Identification (recommended) - Prefer specifying a stable type identifier on your classes to avoid coupling persisted data to Java class names. This makes refactors and package/class renames safer. - Entities: set `@Entity(typeId = "Order")` (choose any stable string meaningful to your domain). - Embedded types: set `@Embedded(typeId = "Address")` likewise. - Morphium uses the `typeId` (when provided) instead of the Java class name to identify the target POJO for incoming data. This decouples stored documents from Java class names and eases migrations when packages or class names change. ```java @Entity(typeId = "Order", translateCamelCase = true) public class Order { /* ... */ } @Embedded(typeId = "Address") public class Address { /* ... */ } ``` #### Notes - `typeId` is available on both `@Entity` and `@Embedded`. - For heterogeneous collections/fields, you can also enable `polymorph = true` to include type information in the stored documents. - Important: set a `typeId` from the beginning. If you first store documents without `typeId`, Morphium will persist the Java class name; after a rename or package move those documents may no longer deserialize. You can set `typeId` in the new version to the old fully‑qualified class name as a recovery step, but it is ugly and potentially confusing—prefer setting a stable `typeId` from day one. ### Renames and Schema Evolution - Field renames: use `@Aliases({"oldName1", "oldName2"})` on the new field to accept legacy field names from MongoDB and in queries during migration. ```java public class User { @Aliases({"name", "user_name"}) private String userName; } ``` - Additional/dynamic fields: add a catch‑all `Map` annotated with `@AdditionalData` to retain unknown fields that exist in MongoDB but not in your POJO. ```java public class User { @AdditionalData(readOnly = true) // set false if you want to write them back private Map extras; } ``` - Combine `typeId` with `@Aliases` and `@AdditionalData` for smoother migrations: keep deserialization working after refactors, accept legacy field names, and preserve unexpected fields. - **`@Aliases` is read/query-only.** It only makes `getField()`/query-key resolution also accept the listed legacy names when reading or building a filter; it never changes what gets written. A `store()` on the new field always writes the canonical field name — a fleet running mixed old/new code will not see the new code's writes show up under the old name, and vice versa. ### Example: Rename Class and Fields Safely #### Version 1 (initial, best practice) ```java // package com.example.v1; @Entity(typeId = "User") // set a stable typeId from day one public class User { @Id private MorphiumId id; private String name; // old field name private int age; } ``` #### Version 2 (after refactor/migration) ```java // package com.example.accounts; // class/package renamed @Entity(typeId = "User") // stable type identifier public class AccountUser { // class renamed @Id private MorphiumId id; // field renamed; accept legacy names from existing MongoDB docs @Aliases({"name", "user_name"}) private String userName; private int age; // capture unknown/dynamic fields to avoid data loss during migration @AdditionalData(readOnly = true) private Map extras; } ``` #### Recovery (if v1 had no `typeId`) - If v1 stored the Java class name, set `@Entity(typeId = "com.example.v1.User")` in v2 so existing documents still deserialize. Then plan a data migration to switch to a clean, stable `typeId` later. #### Notes - Existing documents continue to deserialize because `typeId = "User"` no longer depends on the Java class name. - Legacy documents with `name` (or `user_name` if camelCase translation changed) populate `userName` thanks to `@Aliases`. - Any unexpected fields present in legacy documents are preserved in `extras`. ## Querying ```java // Find one Order o = morphium.createQueryFor(Order.class) .f("customerId").eq("C123") .get(); // Find many with projection/sort List recent = morphium.createQueryFor(Order.class) .f("status").eq("OPEN") .sort("-created") .asList(); ``` ## Field Names (avoid string literals) - Prefer enums over string field names to avoid typos and ease migrations/renames. ```java @Entity(translateCamelCase = true) public class User { @Id private MorphiumId id; private String userName; // stored as user_name public enum Fields { id, userName } } // Safer queries using enums var q = morphium.createQueryFor(User.class) .f(User.Fields.userName).eq("alice"); ``` - Alternative without codegen: use the lambda property extractor helper ```java import static de.caluga.morphium.query.FieldNames.of; var q2 = morphium.createQueryFor(User.class) .f(of(User::getUserName)).eq("alice"); ``` - Enums/lambdas remain stable across refactors and camelCase translation changes. - See How‑To: [Field Names](./howtos/field-names.md) for more options (including annotation‑processor codegen). ## Aggregation ```java var agg = morphium.createAggregator(Order.class, Map.class); agg.match(morphium.createQueryFor(Order.class).f("status").eq("OPEN")); agg.group("$customerId").sum("total", "$amount").count("cnt").end(); agg.sort("-total"); List results = agg.aggregate(); ``` See How‑To: [Aggregation Examples](./howtos/aggregation-examples.md) for more pipelines. ## Write Safety (`@WriteSafety`) `@WriteSafety` on an entity class sets the MongoDB write concern Morphium uses for every write of that type: | `level` | Resulting `w` | Meaning | |-----------------------|---------------------------------------|-------------------------------------------------------------| | `NORMAL` (default) | `0` | fire and forget, only socket errors surface | | `BASIC` | `1` | acknowledged by the primary | | `WAIT_FOR_SLAVE` | `2` | primary plus one secondary | | `MAJORITY` | `"majority"` | MongoDB majority commit, the recommended durable setting | | `WAIT_FOR_ALL_SLAVES` | number of **reachable** members | every member the driver can currently talk to (see caveat) | `timeout` is the `wtimeout` in ms (`0` = wait forever), `waitForJournalCommit` sets `j: true`. Against a standalone server everything above `w:1` is downgraded to `w:1`, because the server could never satisfy it. ```java @Entity @WriteSafety(level = SafetyLevel.MAJORITY, timeout = 5000, waitForJournalCommit = true) public class Order { ... } ``` ### Caveat: `WAIT_FOR_ALL_SLAVES` and member loss `WAIT_FOR_ALL_SLAVES` is a Morphium feature, not a MongoDB write concern: there is no "all members" in MongoDB, so Morphium derives a numeric `w` on every write from the number of replica-set members the driver currently considers reachable. Reachable means the driver's heartbeat has talked to the member and its last contact did not fail. That is what makes the level survive a member loss at all: with a plain `w:3` on a three-member set, one member down means every write waits out `wtimeout` until the member is back. With `WAIT_FOR_ALL_SLAVES` the `w` drops to `2` as soon as the heartbeat has noticed the loss. The price is a **detection window**. Between the moment a member goes away and the first failed heartbeat against it (at most one `heartbeatFrequency`, default 1 s, plus the connect timeout), writes are still sent with the old `w` and block for their `wtimeout`. A member that was never reachable since the client started, or that comes and goes, keeps `w` low. Two consequences to plan for: - Set a finite `timeout`. With `timeout = 0` a write issued inside the detection window waits forever. - Treat a write that ran into `wtimeout` as "durability not confirmed", not as "failed": MongoDB (and PoppyDB) apply the write on the primary and report the missed replication as a `writeConcernError` in an otherwise successful reply. Morphium does not yet raise this as an exception (tracked as [#408](https://github.com/sboesebeck/morphium/issues/408)), so `store()` returns normally in that case. If you need a guarantee that does not depend on client-side topology detection, use `MAJORITY`: it is evaluated by the server against the actual replica-set configuration and tolerates a minority of members being down without any detection window. `WAIT_FOR_ALL_SLAVES` is for the case where you want every *available* copy written and accept the window as the cost. ## Caching - Add `@Cache` to entities to enable read cache; TTL, max entries, and clear strategy are configurable. - Cluster‑wide cache synchronization comes in two flavors — messaging‑driven and watching (change‑stream‑driven); see below for which one to pick. - A JCache adapter is available if you prefer standard javax.cache interfaces. See How‑To: [Caching Examples](./howtos/caching-examples.md) and [Cache Patterns](./howtos/cache-patterns.md) for recipes and guidance. ### Cache Synchronization - Purpose: keep caches consistent across nodes when the underlying data changes. - Mechanism: on a relevant change, other nodes apply a policy from `@Cache.syncCache`: - `CLEAR_TYPE_CACHE`: clear the entire type cache for the entity. - `REMOVE_ENTRY_FROM_TYPE_CACHE`: remove a single entry (by ID) from the cache. - `UPDATE_ENTRY`: re‑read and update the cached entity in place (may briefly expose stale data under concurrent reads—“dirty reads”). - Two independent implementations both drive the same `@Cache.syncCache` policies — pick one, not both, per Morphium instance: **`MessagingCacheSynchronizer`** — hooks into Morphium's own storage listener. Every `store`/`remove`/`update`/`drop` made *through this (or another) Morphium instance* triggers a `cacheSync` message over Morphium's own messaging, which every other node with a `MessagingCacheSynchronizer` attached picks up. ```java var messaging = morphium.createMessaging(); messaging.start(); new MessagingCacheSynchronizer(messaging, morphium); // attach synchronizer ``` - Requirements: messaging must be running (and healthy) on every node that should invalidate its cache. Works against any driver/backend — in‑memory, single MongoDB, PoppyDB, or a replica set — no change‑stream support needed. - Blind spot: it only ever sees writes that go through Morphium's storage listener. Changes made by another application, a raw driver/shell write, a restore, or an admin script are invisible to it and will **not** invalidate remote caches. - Extra strength: this is more than "just a watcher" — the invalidation travels as a regular message on the `cacheSyncType`/`cacheSyncRecord` topics of Morphium's general‑purpose messaging, not a raw DB event. Any node — including a non‑Java, non‑Morphium process that merely understands the message document format — can register its own listener on the same topic and trigger additional logic beyond the built‑in `@Cache.syncCache` handling (e.g. invalidate a different cache layer, fire a notification, write an audit trail). Worth it whenever the desired reaction to a cache‑relevant write is more involved than clear/remove/update. **`WatchingCacheSynchronizer`** — opens a MongoDB Change Stream (`ChangeStreamMonitor`) directly on the watched collections and reacts to whatever it sees change at the database layer, no matter who wrote it. ```java new WatchingCacheSynchronizer(morphium); // no messaging required ``` - Requirements: Change Streams need a replica set (same restriction as `messagingSettings().setUseChangeStream(true)`, see [Messaging](./messaging.md)) — it does **not** work against a standalone/single MongoDB node. Does not need Morphium's messaging running at all. - Strength: catches every change to the underlying collection regardless of the writer — other services, migrations, mongosh, etc. — because it watches the data, not Morphium's write path. - **Which one to use:** - Only Morphium instances ever write the cached collections, and you're on a standalone/single‑node backend without replica‑set/Change‑Stream support (e.g. a lone MongoDB, or you don't want the extra long‑lived watch connection): use `MessagingCacheSynchronizer`. - Cache invalidation needs to be more than the built‑in clear/remove/update policies — e.g. other nodes (possibly non‑Java) should react to the same event with custom logic: use `MessagingCacheSynchronizer`. Its messages are just documents on a known topic, so anything that can read that topic can hook in, not only Morphium instances running `WatchingCacheSynchronizer`. - Other processes/services (not just this Morphium cluster) can write the cached collections directly, or you'd rather not depend on messaging being up on every node, and you have a replica set: use `WatchingCacheSynchronizer` — it invalidates on *any* write to the collection, not just ones that went through Morphium's own listener. - Both can run in parallel if you want belt‑and‑suspenders coverage, but that's usually unnecessary — pick the one that matches your write paths and backend topology. ## Encryption - Annotate sensitive fields with `@Encrypted` and configure providers/keys via `cfg.encryptionSettings()`. ```java cfg.encryptionSettings().setCredentialsEncryptionKey("secret"); // Optional: custom providers // cfg.encryptionSettings().setEncryptionKeyProviderClass(...); // cfg.encryptionSettings().setValueEncryptionProviderClass(...); ``` ## Threading - Async operations run on a dedicated thread pool (virtual threads by default) configured via `threadPoolSettings()`. - Messaging has its own thread pool configuration in `messagingSettings()`. ## Extension Points - NameProvider: dynamic collection naming ```java morphium.setNameProviderForClass(MyEntity.class, (m, cls, def) -> def + "_2025"); ``` - Storage listeners: audit/validation hooks ```java morphium.addListener(new MorphiumStorageAdapter