Surface typed characteristics and pools in Instance JSON encoding

[matt-beanland]

Description

A resource that declares typed characteristics through the unified provider do characteristics do DSL — and pools via pools do — ends up with those records present in the graph as their own per-type resources (labels like :PathCharacteristic, :AssignableCharacteristic), wired to the Instance via :HAS. But the Instance's JSON encoding only picks up the dynamic Diffo.Provider.Characteristic records flowing through instance.characteristics, so the typed records and pool records are absent from serviceCharacteristic / resourceCharacteristic.

defmodule MyApp.Path do
  use Ash.Resource, fragments: [Diffo.Provider.BaseInstance], domain: MyApp.Access

  provider do
    specification do
      id "1d507914-…"
      type :resourceSpecification
      name "path"
    end

    characteristics do
      characteristic :path, MyApp.PathCharacteristic
    end
  end
end

After build, the PathCharacteristic record exists and exposes its own :value calculation; encoding the Instance still produces JSON without a resourceCharacteristic field.

What we need

That what was declared as a characteristic is also visible as one when the Instance is read — directly or through JSON encoding. The DSL declarations (characteristics/0, pools/0) are already exposed at compile time and the records are reachable in the graph; the gap is between "declared and stored" and "visible in the read view."

Why it matters

Typed characteristics and pools were introduced specifically so domain modellers could give characteristic values structure and lifecycle. If those records don't surface alongside the dynamic ones, every Extended domain has to either build the gather-and-merge view themselves or live with a partial round-trip. The risk for downstream consumers is a quiet one — defined values just don't appear — and the rediscovery cost happens once per domain.

There's a real question underneath this about where Diffo's responsibility ends and the consumer's begins (we ended up exploring both sides as we hit this), but landing the observation first feels more useful than leading with the abstract question.

A possible direction

The Util.suppress_rename(:characteristics, …) step is the natural place a wider input could land — characteristics/0 and pools/0 are already the inputs the DSL holds. Worth checking the symmetric story for features (featureCharacteristic).

A bigger-picture thought we drifted toward while exploring this: today Diffo's Provider domain effectively is the TMF638 projection — implicit, partial, and baked in. Making the projection explicit and opt-in (something like a tmf638 do DSL section a consumer declares when they want TMF638-compliant reads) would put the projection in the foreground, let Diffo do the gathering completely when asked, and leave Extended domains free to expose whatever shape they want when not. Equally, the right answer might just be a how-to in the usage rules pointing consumers at the building blocks for their own view. Both feel like valid evolutions — happy either way.

[matt-beanland]

Following on from the original observation — we kept yarning and the shape settled around an IEEE 1164 mental model, which I wanted to capture here in case it's useful.

Three worlds seem to be in play:

• Provider world — vanilla TMF. Knows Characteristic, Relationship, Specification, etc. and their values. (x01)
• Assigner's world — already outside Provider. Adds AssignedRelationship, AssignableCharacteristic, the concept of assignment. Inside Diffo, but a richer world than Provider. (x01z)
• Extended domain world — the consumer's. Adds typed characteristics, pools-as-domain-concepts, calculations, whatever shape they want. (std_logic)

The principle from IEEE 1164: a world's API can only carry the values its world holds. Cross-world surfacing requires an explicit projection — and that projection IS collapse, lossy by nature. The typed PathCharacteristic flattened to {"name":"path","value":{...}} loses the typing, the calculation lineage, the relationships that record carries. Not a defect; that's what collapse is.

1164 names its collapses explicitly: To_X01Z, To_X01, To_StdLogic. Explicit, declared, the loss visible because the conversion is in your face as a function call rather than an implicit cast.

Diffo already has one of these — implicitly. Instance.relationships() is effectively To_TMF638(assignments), folding 2nd-world assignment edges into Provider's serviceRelationship / resourceRelationship. It works; it just isn't named.

That observation reframed the gap for us:

• It's not that Provider should "see more." Provider can only carry its own values.
• It's that the collapses are missing or implicit. AssignableCharacteristic records have no collapse to serviceCharacteristic / resourceCharacteristic. Typed characteristic records (in an Extended domain) have none either. Both are silent for the same reason.
• And the same observation applies inside Diffo: a vanilla Provider consumer who uses the Assigner's pools do would see pools defined but never round-trip to JSON — because the 2nd-world record never collapses to a 1st-world value.

Naming the collapse seems to unlock things:

• Visibility — the projection becomes a first-class concept rather than a hidden customize step.
• Composability — multiple collapses per world (To_TMF638, To_TMF640, etc.), with each one's loss stated up front.
• Symmetry — once To_TMF638(AssignedRelationship) exists explicitly, the sibling To_TMF638(AssignableCharacteristic) is the obvious next step.
• Consumer agency — Extended domains could declare their own collapses (the tmf638 do idea), leaving Provider's surface untouched and giving the consumer naming rights over their richness-to-TMF mapping.

Not asking for any of this — landing it here because the framing helped us, and might be useful in shaping how the gap from the original description gets approached.