Handling new configurations, new versions

[K20shores]

MechanismConfiguration: Versioning and API Redesign

Addressed #266

Proposing a stabilized public API (parse / validate / write returning std::expected), an incremental tool integration roadmap (MIAM → MIEM → TUV-x), and a path to v2. Looking for feedback before opening implementation issues.

---

Summary

Versioning strategy — each version parser translates directly to the
canonical Mechanism type. No chained migrations. Adding a new version
never touches old translators.

Public API — replace UniversalParser / ParserResult with free
functions parse() and validate() returning std::expected<Mechanism, Errors>.
validate() takes by value so parse().map(...).map_error(...) chains with
zero copies. write(m, path) exports to YAML or JSON detected from suffix.

Incremental roadmap — v1.x minor bumps add tools one at a time:

• v1.1: MIAM (aerosol representations + aerosol processes)
• v1.2: emissions inventory (emissions: section, inventory name: on reactions)
• v1.3: TUV-x (photolysis_calculator: path)
• v2.0: mechanism: wrapper, field renames, flat namespace

Emissions linkage — EMISSION reactions carry inventory name: which
matches a source in the emissions: section by name. One source can feed
multiple reactions; the species map handles per-species routing; the
reaction's scaling factor: adds per-reaction scaling on top.

Development parser — repurposed. The
free functions parse() and validate() are wired onto it directly. v1
types remain the public return type (so species_name stays; no break for
current callers); the species_name → name rename is deferred to v2 with
the rest of the field renames.

---

01 — v1 Today

Gas-phase chemistry. Photolysis reactions describe stoichiometry; the rate is
provided externally by the host model. No photolysis calculator integration yet.

Foo        --hv-->   0.8 Bar + 0.2 Baz   (photolysis, rate externally supplied)
Foo + Bar  --k1-->   Baz                  (Arrhenius)
Foo is emitted anthropogenically

mechanism.yaml:

version: 1.0.0
name: My Simple Mechanism

species:
  - name: Foo
    molecular weight [kg mol-1]: 0.05
  - name: Bar
  - name: Baz
    molecular weight [kg mol-1]: 0.034

phases:
  - name: gas
    species:
      - name: Foo
      - name: Bar
      - name: Baz

reactions:
  - type: PHOTOLYSIS
    name: foo photolysis
    gas phase: gas
    reactants:
      - species name: Foo
        coefficient: 1
    products:
      - species name: Bar
        coefficient: 0.8
      - species name: Baz
        coefficient: 0.2

  - type: ARRHENIUS
    name: foo bar reaction
    gas phase: gas
    A: 1.0e-5
    C: 110.0
    reactants:
      - species name: Foo
        coefficient: 1
      - species name: Bar
        coefficient: 1
    products:
      - species name: Baz
        coefficient: 1

  - type: EMISSION
    name: foo emission
    gas phase: gas
    products:
      - species name: Foo
        coefficient: 1

#include <mechanism_configuration/parser.hpp>
#include <mechanism_configuration/v1/mechanism.hpp>
#include <iostream>

int main()
{
  mechanism_configuration::UniversalParser parser;
  auto result = parser.Parse("mechanism.yaml");

  if (!result)
  {
    for (auto& [code, message] : result.errors)
      std::cerr << message << "\n";
    return 1;
  }

  auto* m = dynamic_cast<mechanism_configuration::v1::types::Mechanism*>(result.mechanism.get());
  if (!m)
  {
    std::cerr << "unexpected mechanism type\n";
    return 1;
  }

  std::cout << "Species:    " << m->species.size() << "\n";
  std::cout << "Photolysis: " << m->reactions.photolysis.size() << "\n";
  std::cout << "Arrhenius:  " << m->reactions.arrhenius.size() << "\n";
  std::cout << "Emissions:  " << m->reactions.emission.size() << "\n";

  return 0;
}

---

02 — v1 Stabilized API

Same v1.0.0 chemistry, demonstrating the proposed stabilized public API.

// parser.hpp
std::expected<Mechanism, Errors> parse(const std::string&);
std::expected<Mechanism, Errors> parse(const std::filesystem::path&);

// validate.hpp
std::expected<Mechanism, Errors> validate(Mechanism);  // by value

// exporter.hpp
void write(const Mechanism&, const std::filesystem::path&);

From a config file:

#include <mechanism_configuration/parser.hpp>
#include <iostream>

int main()
{
  namespace mc  = mechanism_configuration;
  namespace mct = mechanism_configuration::v1::types;

  return mc::parse("mechanism.yaml")
    .map([](const mct::Mechanism& m) {
      std::cout << "Species:    " << m.species.size() << "\n";
      std::cout << "Photolysis: " << m.reactions.photolysis.size() << "\n";
      std::cout << "Arrhenius:  " << m.reactions.arrhenius.size() << "\n";
      std::cout << "Emissions:  " << m.reactions.emission.size() << "\n";
    })
    .map_error([](const mc::Errors& errors) {
      for (auto& [code, message] : errors)
        std::cerr << message << "\n";
      return errors;
    })
    ? 0 : 1;
}

Built from structs:

return mc::validate(mct::Mechanism{
  .name    = "My Simple Mechanism",
  .species = { mct::Species{ .name = "Foo", .molecular_weight = 0.05 }, ... },
  .reactions = mct::Reactions{
    .arrhenius = {
      mct::Arrhenius{
        .name = "foo bar reaction", .gas_phase = "gas",
        .A = 1.0e-5, .C = 110.0,
        .reactants = { mct::ReactionComponent{ .species_name = "Foo" },
                       mct::ReactionComponent{ .species_name = "Bar" } },
        .products  = { mct::ReactionComponent{ .species_name = "Baz" } },
      },
    },
  },
})
.map([](const mct::Mechanism& m) { mc::write(m, "out.yaml"); mc::write(m, "out.json"); })
.map_error([](const mc::Errors& errors) { ... return errors; })
? 0 : 1;

---

03 — v1.1: MIAM (aerosol + phase transfer)

Adds aerosol representations: and aerosol processes: sections.

version: 1.1.0

aerosol representations:
  - name: fine mode
    type: SINGLE_MOMENT_MODE
    phases: [ aqueous ]
    geometric mean radius [m]: 5.0e-8
    geometric standard deviation: 2.0

aerosol processes:
  - type: HENRY_LAW_PHASE_TRANSFER
    name: baz partitioning
    gas phase: gas
    gas phase species: Baz
    condensed phase: aqueous
    condensed phase species: Baz_aq
    solvent: H2O
    Henry's law constant:
      HLC_ref [mol m-3 Pa-1]: 1.0e-5
      C [K]: 2500.0
    diffusion coefficient [m2 s-1]: 1.9e-9
    aerosol accommodation coefficient: 0.1

New fields on Mechanism: aerosol_representations, processes.henry_law_phase_transfer.

---

04 — v1.2: MIEM (emissions inventory)

Adds emissions: section. EMISSION reactions gain inventory name: linking to a named source. Multiple reactions can share one source; per-reaction scaling factor: applies on top.

version: 1.2.0

reactions:
  - type: EMISSION
    name: foo emission
    gas phase: gas
    inventory name: foo anthropogenic
    products:
      - species name: Foo
        coefficient: 1

  - type: EMISSION
    name: bar emission
    gas phase: gas
    inventory name: foo anthropogenic    # same source
    scaling factor: 0.5
    products:
      - species name: Bar
        coefficient: 1

emissions:
  data root: /path/to/emissions
  inventories:
    my_anthro_inventory:
      directory: cams/v6.2
      file pattern: CAMS-GLOB-ANT_v6.2_{YYYY}-{MM}.nc
      convention: eccad
  species maps:
    my_mechanism:
      sources:
        my_anthro_inventory:
          - inventory: FOO_EMIS
            mechanism: Foo
          - inventory: BAR_EMIS
            mechanism: Bar
  sources:
    - name: foo anthropogenic
      mode: offline
      type: anthropogenic
      inventory: my_anthro_inventory
      species map: my_mechanism
      temporal interpolation: linear
      vertical injection: surface
      scaling factor: 1.0

Parser validates at load time: every inventory name: has a matching source; every source species map entry references a species in the mechanism.

---

05 — v1.3: TUV-x (photolysis calculator)

Adds a photolysis_calculator: section — just a path to a TUV-x config file. The name-mapping between TUV-x cross-sections and mechanism photolysis reactions lives inside that config.

version: 1.3.0

photolysis_calculator:
  config: path/to/tuvx/config.yml

New field on Mechanism: std::optional<PhotolysisCalculator> photolysis_calculator.

---

06 — v2 Aspirational

End-state: mechanism: wrapper, flat mechanism_configuration:: namespace, processes: key, transforms: co-located on photolysis reactions.

version: "2.0.0"
mechanism:
  species: [...]
  phases: [...]
  aerosol_representations: [...]
  processes:
    - type: PHOTOLYSIS
      transforms:
        - type: equal-weight
          min-wavelength [m]: 2.9e-7
          max-wavelength [m]: 3.5e-7
    - type: EMISSION
      inventory name: foo anthropogenic
    ...

using namespace mechanism_configuration;

return parse("mechanism.yaml")
  .map([](const Mechanism& m) {
    std::cout << m.processes.photolysis.size() << "\n";
  })
  .map_error([](const Errors& e) { ... return e; })
  ? 0 : 1;

---

07 — Versioning Design

Each version parser translates directly to the canonical Mechanism type — no migration chain.

v0 ──────────────────┐
v1 (+ minor bumps) ──┼──► canonical Mechanism
v2 ──────────────────┘

• v0 → canonical: add name = "", gas_phase = "GAS"; convert Phase.species from vector<string> to vector<PhaseSpecies>
• v1 → canonical: 1-1 field map; absent sections leave vectors empty
• v2 → canonical: unwrap mechanism:; rename processes: → reactions struct

The v1 → v2 config migration is handled offline by mechanism-config migrate, not at runtime.

Change	Bump
Restructure flat → sectioned	major
New optional section or field	minor
New optional reaction type	minor
Remove or rename a field	major

---

08 — Implementation Plan

Version	Feature
—	Stabilize public API
1.1.0	MIAM (aerosol)
1.2.0	Emissions inventory
1.3.0	TUV-x photolysis calculator
—	Export
2.0.0	Restructure + renames

Suggested issues (each independently mergeable):

1. parse() + std::expected — replace UniversalParser with free function built on development::Parser; ConfigParseStatus → ErrorCode; v1 types remain the public return type
2. validate(Mechanism) — validate.hpp wrapping development::Parser::Validate(); checks species/phase refs; enables in-code construction pattern
3. MIAM types — SingleMomentMode, TwoMomentMode, HenryLawPhaseTransfer, etc. in v1/types.hpp
4. MIAM parser — read aerosol representations: and aerosol processes:; validate mode phases and species
5. inventory_name on Emission — parse inventory name: YAML key; empty string if absent
6. MIEM parser — EmissionsConfig struct; validate every inventory name: has a matching source
7. photolysis_calculator: parser — std::optional<PhotolysisCalculator> on Mechanism; parse single config: path
8. write() — exporter.hpp; format from suffix; AC: round-trip parse → write → parse produces equal Mechanism
9. v2 restructure + mechanism-config migrate CLI — mechanism: wrapper, flat namespace, species_name → name and other renames; development becomes the v2 parser

[boulderdaze]

Following today’s discussion, I’ve added a list of items we can consider to continue planning the configuration.

• std::expected requires a compiler with C++23 support. We should consider whether this is feasible.

std::expected<Mechanism, Errors> validate(Mechanism); // by value

• The validate() could probably just return Errors.

MIAM (Aerosol)

• We could also use more descriptive keys. For example, C [K] is not clear.

    # Current
    Henry's law constant:
      HLC_ref [mol m-3 Pa-1]: 1.0e-5
      C [K]: 2500.0

    # Suggestion
    Henry's law constant:
      constant [mol m-3 Pa-1]: 1.0e-5
      reference temperature [K]: 298.15
      enthalpy [J mol-1]: 1000.0 (<- not yet divided by gas constant) 
      ( or van't Hoff coefficient) [K]: 2500.0)

MIEM (Emissions)

• We could just use name instead of species name

    # Current
    products:
      - species name: Foo
        coefficient: 1

    # Suggestion
    products:
      - name: Foo
        coefficient: 1

• It looks like data root or directory both refers to a path. I think we should standardize on a single term. Either path or root or directory for consistency.

emissions:
  data root: /path/to/emissions.  <----- path
  inventories:
    my_anthro_inventory:
      directory: cams/v6.2.      <----- also path
      file pattern: CAMS-GLOB-ANT_v6.2_{YYYY}-{MM}.nc
      convention: eccad

   species maps: 
    my_mechanism:   <-- This name doesn't seem standardized. Is this needed? 
                        This species map seems unnecessarily deeply nested
      sources:
        my_anthro_inventory:   <-- This name doesn't seem standardized (name keyword could be used)
          - inventory: FOO_EMIS.
            mechanism: Foo
          - inventory: BAR_EMIS
            mechanism: Bar

  sources:
    - name: foo anthropogenic.  <-- Ok, this is the same as `reactions.inventory name` defined in reactions. 
                  but there is `inventory` keyword below, which is confusing.
      mode: offline
      type: anthropogenic
      inventory: my_anthro_inventory.    <-- here
      species map: my_mechanism
      temporal interpolation: linear
      vertical injection: surface
      scaling factor: 1.0