Skip to content

Tenant Isolation

Jump to bottom
Sietse edited this page Sep 29, 2026 · 3 revisions

Multi Tenant Isolation

Keep one customer's memory unreachable from another's, and see who is crowding whom.
Status ✅ Isolation works. ⚙ Capacity protection is configured, not automatic: see What is manual rather than automatic
Verified cross tenant reads return 404, not 403, on a real server

In plain words

If two companies use the same server, neither may see the other's data. This keeps them apart: one customer cannot find, or even ask for, another's memory.

It also counts what each tenant is doing: how often they find something in memory, and how often their data gets pushed out to make room for someone else. So when one customer complains that things got slow, you can see whether another customer caused it.

⚙ Measuring and enforcing are two separate settings. The counters are on by default; the limits are not. Set a reservation for each tenant you owe capacity to, and the counters tell you which tenants those are.

The everyday example

Two banks share a server. Bank A cannot read Bank B's memory: a request for it simply finds nothing. If Bank A runs a huge job and pushes Bank B's data out, the counters show it, and a reservation set for Bank B in advance keeps the capacity it is owed.

The basics

How it works

  1. Every request says which customer (tenant) it belongs to.
  2. Galahad keeps each tenant's memory separate. A tenant can only find its own memory.
  3. Galahad counts, per tenant, what was found, what was missed and what was pushed out.

How to use it

  1. Choose the mode: single for one organisation, multi when customers must be kept apart.
  2. Pass the customer's tenant_id with every request.
  3. Read the counters to see who uses the memory. Set a reservation if a customer needs guaranteed room.

Setting it up

cfg.tenancy = MERLIN_TENANCY_MULTI;   /* separate trust domains */
merlin_init(&cfg);

With vLLM or SGLang, set GALAHAD_TENANCY=multi.

Every call that reads or writes memory takes a tenant_id. Pass the ID of the customer the request belongs to.

⚠ MERLIN_TENANCY_SINGLE is for one organisation and gives maximum reuse. Use MULTI only when the tenants genuinely do not trust each other. It reduces sharing, which is the point.

⚙ There is no default, and a server that does not state its mode refuses to start:

merlin_init(tenancy=UNSET) -> tenancy mode not configured

A wrong guess would not fail loudly: it would miss every lookup, or serve one tenant's memory to another.

⚠ Choose the mode once. A store written in one mode is refused in the other mode (MERLIN_ERR_TENANCY_MISMATCH).

Optional: a separate keyspace per tenant

merlin_set_tenant_salt(tenant_id, salt);   /* salt: a 64-bit value you choose and keep */

This gives the tenant its own keyspace. Two tenants that send the same prompt then share nothing.

  • It reduces reuse. A system prompt used by 50 salted tenants is stored 50 times instead of once. Use it where tenants are separate organisations, not for teams inside one company.
  • Set it after merlin_init and before the first request for that tenant.
  • Never change it for a tenant that already has data. Its saved memory would no longer be found. Keep the salt in your own configuration.

merlin_get_tenant_salt(tenant_id, &salt) tells you which salt is active (0 = none).

Seeing the pressure

size_t n = 0;
merlin_get_tenant_stats(NULL, 0, &n);            /* how many tenants */

merlin_tenant_stats rows[64];
for (size_t i = 0; i < 64; i++) rows[i].struct_size = sizeof rows[i];
merlin_get_tenant_stats(rows, 64, &n);           /* n can be larger than 64: then the list was cut */
Counter Meaning
tenant_id which tenant this row is about
hits · misses how well this tenant's memory is working
evictions_caused this tenant pushed someone else's data out
evictions_suffered this tenant's data was pushed out
blocks_materialised how much they brought in

⭐ caused - suffered is net pressure on other tenants. When a tenant pushes out its own data, both counters go up, so the subtraction shows pressure on others rather than total churn.

Reading the counters is cheap enough to do on a busy path. merlin_reset_tenant_stats() sets them back to zero, for example to measure one time window. Saved memory is not touched.

Protecting a tenant

merlin_reserve_tenant_vram(tenant_id, bytes);   /* guaranteed capacity */

⚙ The default reservation is zero, which means capacity is shared freely until you say otherwise. A multi tenant deployment that owes anyone guaranteed capacity should set a reservation per tenant at startup.

  • If all reservations together are larger than the memory Galahad may use, the call returns MERLIN_ERR_CAPACITY.
  • bytes = 0 removes the reservation.

What is manual rather than automatic

⚙ Protection is something you configure. There is no automatic defence against a noisy neighbour: an operator who sets reservations is protected, one who does not is not.
✅ built in tenant separation, per tenant counters, reservations, tenant erasure
⚙ configure it yourself set reservations in advance for tenants you owe capacity
⚠ not available per tenant memory use in bytes: merlin_tenant_stats has no bytes_resident field. Use blocks_materialised as a stand-in

⚠ The counters do not change what gets evicted. They are measurement.

⭐ In one line: reservations protect capacity; the counters tell you who needs one.

Erasing a tenant

size_t removed = 0;
merlin_forget_tenant(tenant_id, 1, &removed);

This is the GDPR call. It removes the tenant's saved entries. With erase_payloads = 1 it also deletes the tenant's saved files. With 0, the files stay on disk.

With Encryption at Rest, you can also destroy the tenant's key in your key manager. The tenant's data can then no longer be read.

Isolation, proven

Through a real server, with the Memory Inspector:
tenant 1 reads its own entry 200
tenant 0 asks for the same entry ⭐ 404, not 403
admin reads it 200
every sub resource (/tensors, /lineage) 404 cross tenant
a misspelled permission startup failure, exit 2

⭐ Why 404 and not 403. A 403 would confirm the entry exists. The tenant check is made on the server, never taken from the request.

Troubleshooting

Symptom Cause Fix
tenancy mode not configured cfg.tenancy left at zero, or GALAHAD_TENANCY not set set SINGLE or MULTI
MERLIN_ERR_TENANCY_MISMATCH at startup the store was written in the other mode use the original mode, or a new store directory
a tenant's lookups all miss its salt changed restore the original salt
one tenant starves another no reservation set merlin_reserve_tenant_vram; there is no automatic defence
evictions_caused high for one tenant they are crowding others set reservations for the tenants you owe capacity
you want per tenant memory use in bytes merlin_tenant_stats has no bytes_resident field use blocks_materialised as a stand-in

Related

Clone this wiki locally