Define semantics and implement CopyDirective.add

[amc-corey-cox]

`CopyDirective.add` is declared in `transformer_model.yaml` (range: Any) but has no description and is not read anywhere in `schema_mapper.py`.

Needs:

• Decide what `add` means relative to `copy_all`/`exclude`/`exclude_all`/`include`. The other four operate on existing sub-elements of the source; `add` presumably injects new ones not present in the source.
• Define the value shape (full element definitions? references?).
• Document and implement in the `SchemaMapper.copy*` methods.

Spun out of #67 / #244.

[github-actions]

Summary & Analysis

Context

This issue was spun out of the PR #244 (which fixes exclude/exclude_all bugs on slot-list classes) and the older issue #67 (user wanted to remove an inherited slot constraint like pattern when copying to a derived schema).

Current State of CopyDirective.add

In transformer_model.yaml (line 757):

add:
  range: Any
  # (no description)

In schema_mapper.py, _copy_dict and _copy_list handle copy_all, exclude, exclude_all, and include — but add is never read. It is also absent from any tests.

Semantics Question

The four implemented directives all operate on sub-elements that already exist in the source schema:

Directive	Effect
copy_all	Seed the copy-set with every source sub-element
exclude	Remove named sub-elements from the copy-set
exclude_all	Clear the entire copy-set
include	Re-add named sub-elements from the source back into the copy-set

add is the obvious complement: inject new sub-elements that are not present in the source at all. This is a fundamentally different operation — it can't just reference a source name, it needs to carry the full element definition.

The natural processing order would extend the existing sequence:

1. copy_all — seed from source
2. exclude — prune from copy-set
3. exclude_all — clear copy-set
4. include — restore named source elements
5. add — inject wholly new elements (not from source)

Design Choices to Resolve

Shape of add values

Option	Pros	Cons
List of SlotDefinition-like objects (typed as Any)	Flexible; parallel to SlotDerivation.target_definition	Parsing complexity; Any range gives no IDE help
List of strings (names only)	Simple	Carries no metadata — can't define new elements without metadata
Delegate entirely to SlotDerivation without a populated_from	Reuses existing machinery; already documented as the path for override use cases	Less discoverable; CopyDirective.add would remain vestigial

The most pragmatic path is likely option 3 for the override-slot-attributes use case (already noted in #244) and option 1 for CopyDirective.add when you genuinely need to inject brand-new sub-elements. A SlotDerivation with no populated_from and a target_definition handles "copy this slot but clear its pattern"; CopyDirective.add handles "add a slot that doesn't exist in the source at all."

Scope of add

• For _copy_class: should it support adding new attributes only, or also new named slots?
• For _copy_schema: should it support adding new classes, slots, enums, types?

Relationship to SlotDerivation.target_definition

Issue #244's description explicitly notes:

Mechanism for overriding/clearing individual attributes of a copied slot (the use case described in #67's body) is a different feature, likely belonging on SlotDerivation.target_definition.

So the scope split is:

• CopyDirective.add → add entirely new sub-elements to the derived schema that have no source counterpart.
• SlotDerivation.target_definition (no populated_from) → copy a source slot but override some of its attributes in the target.

Suggested Implementation Path (once semantics are agreed)

1. Update transformer_model.yaml: add a description to add, clarify the range (e.g., a multivalued list of SlotDefinition-equivalent), and add it to the processing-order docstring in CopyDirective.
2. Regenerate transformer_model.py via make src/linkml_map/datamodel/transformer_model.py.
3. Implement in schema_mapper.py: add add handling at the end of _copy_dict and _copy_list (after include), inserting the new element definitions into the target collection.
4. Tests: parametrized tests in the scaffold pattern (tests/README.md) covering: add a new slot to a class, add with copy_all, add with exclude_all (to get a target with only the added slots).

Open Design Question for Chris

The most important thing to decide before coding is: what is the expected YAML serialization of an add value? For example:

# Option A: inline slot-definition-style dict
copy_directives:
  - element_name: MyClass
    copy_all: true
    add:
      - name: new_slot
        range: string
        required: true

# Option B: reference to a SlotDerivation (reuse existing machinery)
slot_derivations:
  new_slot:
    # no populated_from → pure injection
    target_definition:
      range: string
      required: true

Option B may already be implementable without touching CopyDirective.add at all — worth checking whether SchemaMapper currently handles SlotDerivation entries with no populated_from.