Release 1.12.0: Compound Documents rework: in-process includes, finer-grained config, stronger security and descriptive meta

[MoonWorm]

Overview

1.12.0 is a rework of the Compound Documents plugin and the standalone resolver behind it. Four headline changes:

• In-process includes. A resource type the app serves itself is no longer fetched by the app calling itself over HTTP: the read runs through the framework directly, as the client's principal, with validation and access control applied as before. No socket, no JSON written and parsed again, no second worker thread held while the original request waits — roughly 1.8× faster per request on a 3-level include.
• Finer-grained configuration. Every per-type setting lives in one mapping entry: the service url, maxBatchSize, whether the service is trusted with the client's credentials (propagateCredentials), and — for types the app serves itself — the transport, IN_PROCESS or HTTP. New switches decide how strict the plugin is: deduplication, unsupportedIncludes, credentialHeaders. All of it is validated at startup, with errors pointing at the exact key.
• Stronger security. Only headers that are safe to forward reach downstream services, the client's credentials go only to services you trust with them, every part of a downstream URL is percent-encoded, and same-app includes no longer depend on the client's Host header.
• Descriptive meta. A response whose included is missing resources now says so: meta.includedIncomplete names each gap and why — a failed fetch, a type with no route, a limit reached, an unsupported path. And an include path the server can't serve is a 400 whose meta.path names it, instead of being dropped silently.

Underneath, includes now follow the spec: only what the client asked for ends up in included, and a primary resource is never repeated there.

Beyond the plugin: a Swagger UI next to Redoc on the live OpenAPI sample, a new Showcase page listing projects built with JsonApi4j, and JSON:API member names used through constants across the codebase.

This release contains breaking changes. They are listed first.

Breaking Changes

Configuration: one mapping entry per resource type

Per-type settings were spread over mapping and batchSizeMapping. They now live in one entry per type:

1.11.0	1.12.0
jsonapi4j.cd.mapping.<type>=<url>	jsonapi4j.cd.mapping.<type>.url=<url>
jsonapi4j.cd.batchSizeMapping.<type>=<n>	jsonapi4j.cd.mapping.<type>.maxBatchSize=<n>
jsonapi4j.cd.deduplicateResources=true	jsonapi4j.cd.deduplication=DATA_AND_INCLUDED (the default)
jsonapi4j.cd.deduplicateResources=false	jsonapi4j.cd.deduplication=NONE

# before
jsonapi4j:
  cd:
    mapping:
      orders: https://orders.internal/jsonapi
    batchSizeMapping:
      orders: 50

# after
jsonapi4j:
  cd:
    mapping:
      orders:
        url: https://orders.internal/jsonapi
        maxBatchSize: 50

Same-app includes no longer pass through servlet filters

Read in-process, an include of a type the app serves itself is subject to validation and the Access Control plugin — but not to servlet filters: URL-based security rules, request logging or metrics no longer see include requests. Where they must, fetch those types over HTTP from the app's own address:

jsonapi4j:
  cd:
    mapping:
      default:
        url: http://127.0.0.1:8080/jsonapi
        transport: HTTP

Credentials reach other services only when you say so

In 1.11.0 the client's Authorization, Cookie and X-Authenticated-* headers were copied onto every include call — partner and third-party APIs included. Now a type mapped to another service receives them only with propagateCredentials: true:

jsonapi4j:
  cd:
    mapping:
      orders:
        url: https://orders.internal/jsonapi
        propagateCredentials: true    # your own service: forward the caller's identity
      rates:
        url: https://partner.example.com/jsonapi   # a partner: no credentials

Types served by the app itself always get the client's identity.

Responses change

• included holds only what was requested. Includes were merged per resource type rather than per path: with include=citizenships,placeOfBirth.currencies, the citizenship countries' currencies were fetched and returned too. Clients relying on those extra resources must ask for them.
• A primary resource is no longer repeated in included when an include path cycles back to it (/users/1?include=relatives.relatives). Set deduplication: INCLUDED_ONLY to repeat reached primary resources in included, so every relationship resolves from included alone.
• An unknown include name is a 400 — include=pets on a type without a pets relationship — in every JsonApi4j app, with or without the CD plugin.
• An include path deeper than maxHops is a 400 instead of being cut short silently. unsupportedIncludes: IGNORE restores the lenient behaviour, now with the cut reported in meta.

Headers no longer forwarded downstream

The client's Accept, Accept-Encoding, conditional headers (If-None-Match, If-Modified-Since, If-Match, If-Unmodified-Since, If-Range), Range, hop-by-hop headers, and Forwarded / X-Forwarded-For / X-Real-IP are no longer copied onto include calls. Each of them either broke the fetch — a gzipped body, a 304, a 206 — or let a caller choose what the downstream service sees.

Error handling moved to jsonapi4j-base

The error handling API moved from pro.api4.jsonapi4j.servlet.response.errorhandling to pro.api4.jsonapi4j.errorhandling. Update the imports of any custom ErrorHandlerFactory.

Plugins must be registered (plain Servlet API)

In a plain servlet setup, a CD or OpenAPI plugin enabled in the config file but missing from the PluginRegistry used to run anyway, from a second copy of its configuration. It is now inactive, with a warning naming the cause. Spring Boot and Quarkus always register the plugin.

Standalone resolver API

For those embedding jsonapi4j-compound-docs-resolver directly, e.g. in an API gateway:

1.11.0	1.12.0
new DomainSettings(url, maxBatchSize), DomainSettings.of(url)	DomainSettings.overHttp(url, maxBatchSize, propagateCredentials), DomainSettings.inProcess(maxBatchSize)
DefaultDomainSettingsResolver.from(...)	new DefaultDomainSettingsResolver(Map<String, DomainSettings>)
CompoundDocsRequest headers: Map<String, String>	Map<String, List<String>>, case-insensitive
CompoundDocsResolverConfig	gains credentialHeaders
CachingCompoundDocsFetcher, BatchFetchResult, HttpFetchResult	BatchFetcher decorators, FetchResult — see Standalone resolver
ExecutorService arguments	Executor — existing callers still compile

Constants on relationship documents

ToOneRelationshipDoc and ToManyRelationshipsDoc no longer declare INCLUDED_FIELD and JSONAPI_FIELD; use BaseDoc.INCLUDED_FIELD and BaseDoc.JSONAPI_FIELD, which now also holds DATA_FIELD.

Compound Documents

How includes are fetched

Each included resource type is fetched hop by hop, in filter[id] batches:

1. Mapped to a url: over HTTP from that service. Set it only for types another service serves.
2. Otherwise the app serves the type itself, and reads it in-process — through the framework, as the client's principal, without an HTTP request.
3. Unless its transport is HTTP: then it's fetched over HTTP from mapping.default.url, the app's own address, through every servlet filter and with the client's credentials.

transport is set per type, or for all same-app types on the reserved default entry — so either side can be the exception:

jsonapi4j:
  cd:
    mapping:
      default:
        url: http://127.0.0.1:8080/jsonapi
        transport: HTTP           # same-app types over HTTP...
      users:
        transport: IN_PROCESS     # ...except users

Or through environment variables, e.g. JSONAPI4J_CD_MAPPING_DEFAULT_URL and JSONAPI4J_CD_MAPPING_DEFAULT_TRANSPORT=HTTP. Startup fails on a transport that can't work: HTTP without mapping.default.url, or IN_PROCESS on a type mapped to another service's url.

Why in-process is the default — and the better choice wherever it applies:

	In-process	Over HTTP
Per include fetch	a method call	a socket, every servlet filter, JSON written and parsed again
Worker threads	none extra	a second one held while the original request waits — under load a bounded pool can run out
Reachability	always	the app's own URL must be reachable from itself: TLS, bind address, mTLS, graceful shutdown
Tests	works with MockMvc and RANDOM_PORT	needs a real, known port

In a rough local benchmark — Spring Boot sample, in-memory data, include cache off — a 3-level include was ~1.8× faster per request in-process (0.90 ms vs 1.64 ms) at ~1.6× the throughput; without includes both performed the same. With a real database behind your operations the relative gain shrinks; the threads not taken remain.

Two things to know:

• In-process results skip the include cache. They are cheap to read and always fresh; the cache serves responses of other services.
• The executor is used re-entrantly: included resources are fetched on the framework's executor, and an in-process read resolves relationships on that same executor. The default cached thread pool handles that; a bounded executor you provide has to be sized for both.

One mapping entry per resource type

jsonapi4j:
  cd:
    defaultMaxBatchSize: 20
    mapping:
      orders:                                    # served by another service
        url: https://orders.internal/jsonapi
        maxBatchSize: 50
        propagateCredentials: true
      rates:
        url: https://partner.example.com/jsonapi # no credentials for a partner API
      users:
        maxBatchSize: 100                        # served by this app, fetched in-process

Validation at startup points at the exact key: an entry needs a url, a maxBatchSize or a transport; a url must be an absolute http(s) URL; maxBatchSize must be positive; propagateCredentials without a url is rejected; and default is reserved — it takes only url and transport, and a resource type named default fails startup.

Header propagation

With propagation: HEADERS (the default), include calls carry the client's headers — but now only those that are safe to forward, with every value of a repeated header (cookies split across HTTP/2 lines are joined into one Cookie header).

Credentials are a separate list, jsonapi4j.cd.credentialHeaders. The default covers Authorization, Cookie, Proxy-Authorization and the X-Authenticated-* principal headers. If your clients identify themselves another way — an X-Api-Key, or the headers a custom PrincipalResolver reads — add them, so they are treated as credentials too:

jsonapi4j:
  cd:
    credentialHeaders: [Authorization, Cookie, Proxy-Authorization, X-Api-Key]

Unsupported include paths

JSON:API requires 400 Bad Request for an include path the server can't serve. JsonApi4j now checks every read request against the relationships of the requested type. Each server checks its own types, so no shared schema is needed: when a later segment names a relationship that doesn't exist, the service owning that type rejects it, and the resolver maps the name back to the client's full path.

{ "errors": [ {
  "status": "400",
  "code": "UNSUPPORTED_INCLUDE",
  "detail": "Resource type 'countries' has no relationship 'economy'",
  "source": { "parameter": "include" },
  "meta": { "path": "placeOfBirth.economy" }
} ] }

jsonapi4j.cd.unsupportedIncludes decides what the CD plugin does with such paths — unknown names and paths deeper than maxHops alike:

• FAIL (default): the 400 above, one error object per path.
• IGNORE: a 200 with everything that could be resolved, each unsupported path listed in meta.includedIncomplete.

Failed includes and partial responses

jsonapi4j.cd.errorStrategy now behaves consistently across every kind of failure: a non-2xx response, a timeout, a connection error, a response that isn't JSON:API, a type with no route.

• IGNORE (default): failures are isolated per filter[id] batch, and the response tells the client what's missing and why:

  "meta": {
    "includedIncomplete": [
      { "reason": "FETCH_FAILED", "type": "currencies" },
      { "reason": "MAX_INCLUDED_RESOURCES", "path": "relatives.relatives" }
    ]
  }

  Reason	Meaning	Retry helps?
  FETCH_FAILED	a fetch of the type failed	likely
  NO_ROUTE	no route to the type's service	no — server configuration
  MAX_INCLUDED_RESOURCES	included reached maxIncludedResources before the path was resolved	no
  UNSUPPORTED_INCLUDE	the path is unknown or deeper than maxHops (with unsupportedIncludes: IGNORE)	no

  A response missing resources because a fetch failed or a type has no route is marked Cache-Control: no-store, so no shared cache keeps it for the full max-age. Limits and unsupported paths give the same result on every request, so those responses stay cacheable.

• FAIL: a JSON:API error document — 504 for a timeout, 502 for another downstream failure, 500 for a type with no route — with no downstream URLs in the details.

Errors go through the same error handler registry as every other API error. Plugins now contribute their own mappings through JsonApi4jPlugin#errorHandlerFactory(), which never override yours.

Standalone resolver

The resolver now fetches through a chain of small BatchFetcher decorators, one concern each:

OverHttp  → Caching → Chunking → ErrorStrategyAware → HTTP client
InProcess →           Chunking → ErrorStrategyAware → in-process fetcher

DomainSettings has two kinds of route, DomainSettings.overHttp(url, maxBatchSize, propagateCredentials) and DomainSettings.inProcess(maxBatchSize). Routing comes from jsonapi4j.cd.mapping by default; replace it with your own DomainSettingsResolver when routes live elsewhere — for example in a service registry such as Consul:

@Bean
public DomainSettingsResolver domainSettingsResolver(ServiceRegistry registry) {
    return resourceType -> registry.find(resourceType)
            .map(service -> DomainSettings.overHttp(service.jsonApiUrl(), 50, service.isInternal()));
}

Whatever it returns Optional.empty() for is treated as served by the app itself and fetched per its transport.

The include cache is in-memory by default (cache.enabled, cache.maxSize); for a cache shared between instances, provide a CompoundDocsResourceCache — for example one backed by Redis. A gateway can also reject an unsupported include path before proxying anything, with IncludesChecker.from(config).

Also fixed

• Downstream URLs are percent-encoded. A space or non-ASCII character in a custom query parameter or an id failed the request with a 500, and a decoded & or = could inject parameters such as a different filter[id]. Multi-valued parameters are sent as repeated parameters, and propagated custom query parameters are part of the cache key.
• The primary response is captured reliably: decoded as UTF-8 regardless of the JVM's default charset, with text written through getWriter() flushed before reading, and an exact Content-Length on the rewritten body.
• Same-app includes no longer use the client's Host header. In 1.11.0 the app's own base URL was built from it, so a caller could point include requests at another host, and the response could be cached for later callers. Upgrading is recommended for anyone running the CD plugin.
• maxIncludedResources stops resolution only when included has reached the limit and resources are still left to fetch; a hop that has started always finishes.
• On relationship endpoints, only include paths starting at that exact relationship apply — relativesX is no longer treated as being under relatives.
• The CD filter and the OpenAPI servlet take their configuration from the registered plugin and JsonApi4j, not from a second copy read from servlet-context attributes.

Other Changes

• Swagger UI on the live OpenAPI sample, next to Redoc: https://api4.pro/oas-example/
• Showcase: a page listing projects built with JsonApi4j — https://api4.pro/showcase/. Running JsonApi4j in production? Submit your project through https://api4.pro/contact/ or GitHub Discussions.
• JSON:API member names are referenced through constants across the codebase, now gathered on BaseDoc for the top-level members.
• Expanded documentation: how includes are fetched, header propagation, unsupported include paths, incomplete included, error handling and performance.

Migration Checklist

1. Rename CD keys: mapping.<type> → mapping.<type>.url, batchSizeMapping.<type> → mapping.<type>.maxBatchSize, deduplicateResources → deduplication.
2. Set propagateCredentials: true on every mapped service that needs the caller's identity — your own services, typically not partners.
3. If URL-based security rules or other servlet filters must see same-app include requests, set mapping.default.url and transport: HTTP.
4. If your clients identify themselves with headers beyond the defaults, add them to credentialHeaders.
5. Check clients against the response changes: only requested resources in included, no repeated primary resources, 400 for unknown or too deep include paths.
6. Update imports of custom error handlers to pro.api4.jsonapi4j.errorhandling.
7. Plain Servlet API: register the CD and OpenAPI plugins in the PluginRegistry.
8. Embedding the standalone resolver: switch to DomainSettings.overHttp(...) / inProcess(...) and the other API changes above.
9. If you provide a bounded executor, size it for in-process includes too.

Resources

• Documentation: https://api4.pro/
• Compound Documents plugin: https://api4.pro/compound-docs-plugin/
• Standalone resolver: https://api4.pro/compound-docs/
• Performance: https://api4.pro/performance/
• Error Handling: https://api4.pro/error-handling/
• Configuration: https://api4.pro/configuration/
• Release notes: https://github.com/MoonWorm/jsonapi4j/releases/tag/v1.12.0