Pluggable simulator policies: register external Policy classes without editing Registry.java

[yanivholder2]

Hi @ben-manes — floating this as a question before any PR. Would you accept a small extension point letting the simulator load Policy implementations that live outside the caffeine source tree, and if so, do you have a preferred mechanism? Happy to implement and sign the CLA.

Context

We run a research framework that grades cache-replacement policies on hit-rate using caffeine's trace simulator as the ground-truth oracle. Our candidate caches are implemented in Rust and bridged to the simulator via JNI — each is a Policy subclass that loads a native library and forwards record/stats.

Today the only way to wire a new policy into the simulator is to edit Registry.buildRegistry() to add a register*() call. That forces downstream projects to maintain a fork and rebase as the registration chain evolves. Everything else we need is already drop-in: our policy classes live in new packages (no file conflicts) and all settings are passed as -D overrides at runtime.

What's currently closed

• Registry.buildRegistry() hard-codes the set of register*() calls; registerMany(...) stores factories in the private factories map keyed by the @PolicySpec name.
• policies() resolves names from settings against that map, erroring if absent.
• No ServiceLoader / META-INF/services, no classpath scanning, no config key for external policies — so there's no public hook to extend factories.

The Policy contract itself is already self-describing (@PolicySpec carries name() + characteristics(); policies construct from a Config), so an external class needs no new contract — just a discovery path.

Two candidate mechanisms (~30 LOC either way, backwards-compatible, default off)

A. Config + reflection. A caffeine.simulator.external-policies = [ "<fqcn>", … ] list (default empty in reference.conf, read by BasicSettings), loaded by a registerExternal() called at the end of buildRegistry():

private void registerExternal() {
  for (String fqcn : settings.externalPolicies()) {
    var policyClass = Class.forName(fqcn).asSubclass(Policy.class);
    registerMany(policyClass, config ->
        Set.of(policyClass.getConstructor(Config.class).newInstance(config)));
  }
}

Selection stays by @PolicySpec name. Fits the existing Typesafe-Config style; a bad FQCN fails loudly at startup, consistent with unknown-policy errors in policies().

B. ServiceLoader. Discover via ServiceLoader.load(Policy.class) and register those annotated with @PolicySpec. Zero runtime config; most "standard Java." Cost to consumers: shipping a META-INF/services entry.

Why we lean toward A

We pass all simulator settings as -D overrides, so a config list means the policy FQCN comes straight from our runner with nothing assembled inside the caffeine checkout. ServiceLoader would require generating/merging a META-INF/services file in the clone — the one "edit a file in caffeine" step we're trying to remove. That said, we're glad to implement whichever you prefer.

Scope / compatibility

Default empty → existing runs byte-for-byte unaffected. No new dependencies (stdlib reflection or ServiceLoader). Touches Registry, BasicSettings, reference.conf (A) or adds ServiceLoader discovery (B).

[ben-manes]

Hi @yanivholder2!

My main concern is keeping it simple and approachable. Other than myself, the simulator is used by graduate students conducting research or senior developers with little Java experience implementing caches in their preferred language. Therefore less magic to understand and an obvious setup is ideal. The policies are each self-contained attempts to be a reference algorithm if someone wanted to implement it for their own caching library. As these users have a short-term interest their changes don't tend to be upstreamed, as it's effort and it shifts the maintenance burden onto me.

With that in mind, forking tends to be necessary anyway. If a researcher is investigating a new idea it likely relies on a different type of signal that is not modeled currently, such as the miss penalty (aka. latency, cost) or prefetching. Then they need to modify the AccessEvent and add support for new trace formats. For them to not fork would require more pluggability and clarity on how to build externally with their extensions. I don't think most would want that even with AI assist because they'll have to learn Java debuggers, etc to step through code rather than scatter print statement additions.

I would lean towards (A) as well since it's less magical, solves your problem, is simple. Your code snippet could be simplified to register, but itself identifies a quirk that registerMany is useful to provide multiple configuration variants at once. That would lean towards using a factory instead, e.g. the static factory methods (public static Set<Policy> policies(Config config)) currently in use. Perhaps a new annotation, e.g. @Policy.Factory, could be on the constructor or method as a marker for how to instantiate it.