0.9.0

0.9.0 — 2026-10-24

This file accumulates everything landed since 0.7.7. There is no 0.8.0: that
line was the cross-firmware relocation experiment, and it was dropped on purpose —
moving a calibration between firmware revisions is only safe with a professional
tuner validating the result, and without one the output would be dangerous by
default (the reasoning is in the README roadmap). Three things changed shape:

• the apply side — a new engine with an explicit identity policy, built on the
  schema 5.0 recipe specification it reads, exposed as a ladder of four commands
  (tune-strict → tune-shift → tune-unique → tune-any) with a matching pair
  of producers (cook-strict / cook-forced). The engine was verified against the
  flag-driven tune byte-for-byte on 47 real pairs before that command was
  deleted, which is why the old code is gone rather than deprecated;
• the TUI — the 2 625-line single module was retired and rebuilt screen by
  screen on top of the API gate: BOOT, HOME, IDENTIFY, INSPECT, MAPS, COOK, DIFF,
  APPLY, BATCH, CONTRIBUTE and HELP;
• the contribution wizard — a screen, a wider dump intake, and a generated
  extractor that now measures the identity instead of leaving four TODOs.

Added

• The DIFF screen — what changed between two dumps, table by table. COOK
  answers "make this pair portable"; DIFF answers the question that comes first,
  and it is now a screen of its own (HOME card 3) with the pair shared in both
  directions: [D] in COOK hands the two files over, [C] in DIFF hands them
  back. The summary never lets the explained part stand in for the whole — the
  reference pair has 34 changed tables and 69 changed regions no table accounts
  for, and both are shown, with [U] listing the second and [⏎] rendering
  both sides of a region byte by byte. [D] opens the selected table as
  stock / tuned / Δ per cell against its measured breakpoints (a view the CLI
  cannot give at all); [P] writes both sides as SVG using those same
  breakpoints; [X] writes the delta as long-form CSV with breakpoints. Filters
  (changed / realigned / no axis / all) keep exactly what their labels
  say, and axis-backed tables rank above inferred ones — a no axis table's
  -100% is computed against row shapes the scanner guessed and must not lead.
  A region's size is printed as "largest N B", never as a changed-byte total: a
  region is an extent, and a number that cannot be reconciled is worse than two
  honest ones. Nothing is named anywhere on the screen.

• The product is now tested off Linux. Wheels were built for macOS and
  Windows, but the suite had only ever run on Ubuntu — so "the TUI works on
  Windows" was an assumption. A portability job now runs the TUI and console
  tests on windows-latest and macos-latest. It is advisory (continue-on-error)
  until it has been green twice, because the suite was written on Linux and the
  first run is worth reading rather than obeying.

• Console output is UTF-8 by default. On Windows a redirected stream uses the
  ANSI code page, so openremap identify x.bin > out.txt raised
  UnicodeEncodeError on the first em dash — and the copy is full of →, ✓,
  ⚠, Δ on purpose. openremap.console.force_utf8() reconfigures stdout/
  stderr (with errors="replace", so a glyph a terminal cannot draw becomes
  ? instead of killing the command) and is a no-op on a UTF-8 stream. Wired into
  the CLI entry point and the JSON-RPC transport; the TUI's own terminal handling
  is left alone.

• The picker lists the drives on Windows. A tree rooted at C:\Users\…
  can only climb as far as C:\ — no arrow key leaves a drive — so the drives are
  named in the header and typed into the path field (D:, normalised to D:\).
  No jump key is invented: in this screen a printable key belongs to the path
  field. The row is hidden off Windows, where drive letters are not a concept.

• CI runs the TUI off Linux — and that is how these were found. The new
  portability job (windows-latest + macos-latest) caught two real defects in the
  picker on its first run, plus one test that only worked because the corpus was
  present locally. D: is rewritten to D:\\ only when it is a bare drive spec;
  the first version also rewrote UNC paths. The rule moved into
  normalize_typed() — pure text in, text out — because two tests in that file
  asserted str(_typed_path(...)), which runs through WindowsPath and therefore
  measured the platform, not the rule. The drive row's test asserted "hidden" — true on POSIX,
  false on Windows — and now forces both directions. And TestClearing in
  test_recent.py used the corpus without the guard its siblings had, so it failed
  on every platform where tests/data/ is absent, which is all of CI.

• The file picker's path field stopped failing silently. Typing a folder that
  does not exist left the tree exactly where it was, with no message at all —
  against the picker's own stated rule that a rejected file is always named. It
  now says no such folder: … and stays open. The same path also stops mangling
  the Windows idiom for changing drives: D: is drive-relative there (meaning
  "the current directory on D:", often not a directory at all), so it is read as
  D:\ — while on Linux and macOS the identical text is a filename and is left
  alone.

• hex_region — a byte range of a binary as hex, so the TUI never opens a
  file itself. map_image accepts the caller's breakpoints (x_axis /
  y_axis) instead of re-deriving them from a descriptor that may not carry the
  offsets: the DIFF screen already measured them, and the picture must not fall
  back to cell indices while the grid next to it shows real breakpoints.

• The apply side says out loud that it does not correct checksums. Every
  apply command printed APPLIED, a sha256 and the bytes written — all true, and
  all about the recipe. Nothing in this engine rewrites a stored checksum, so
  the written file is expected to be stale wherever the family carries one, and
  the operator learned that from the ECU instead of from us. The report now
  carries a measured verdict on the bytes it produced
  (report.result.checksums: ok / fail / skip), shown by the CLI and by the
  TUI's CHECKSUMS row, together with the command that re-checks the written
  file. It is a disclaimer, never a refusal: a stale checksum does not make the
  recipe wrong, it makes the file unflashable. skip is the honest answer for a
  family whose checksum we have never cracked (Bosch EDC17 among them) and is
  never presented as a pass. Cost: one detector pass, 22–62 ms on a 4 MB dump.
  Every successful apply also prints a standing disclaimer, set apart from the
  check rows: OpenRemap does not correct checksums automatically, a tune that
  will run in a real vehicle must be reviewed by a dedicated tuner shop, and the
  supported use is right to repair and track racing — not modifying an everyday
  car. It is a constant in the report (result.disclaimer), so the CLI and the
  TUI cannot drift apart on it.

• The wizard now says what your dumps can prove. A marker is "stable across
  your dumps", and that is evidence of a family signature only if the dumps are
  different calibrations — two dumps of one calibration cannot, with any
  algorithm, tell a family token from that calibration's version string. Measured
  on two real families: Bosch EDC17 with two dumps of one calibration produced an
  extractor covering 0 of 17 known dumps; two dumps of different calibrations
  produced one covering 14 of 17, exactly the set the hand-written marker
  covers. Denso SH7058: 22/35 from one model year with a marker made of
  repeated data, 35/35 from two dumps six years apart using the ECU's own OS
  banner. So analyze now reports calibration_diversity
  (same-calibration / different-calibrations / near-identical /
  distinct-builds, by identity when the family is known and by shared bytes when
  it is not), the CLI and the CONTRIBUTE screen print it before the candidates,
  and the PR description gains a What your dumps can prove section. The intake
  copy changed with it: the advice was "two or more dumps", which counts dumps
  when what matters is that they are different calibrations.

• A repeated data pattern can no longer lead the candidate ranking.
  signatures.repeated_motif() flags a run built from one short motif (≥6
  repetitions over ≥60% of its length, any alignment): the measured
  'gCaCA$3:f3F$&j4j3j#WD$3TD$3TD…' — 152 printable bytes, 120 of them $3TD —
  ranked first and was recommended. It stays in the list with its reason, like
  every other generic run, but it can no longer be the recommendation.

• The contribution wizard measures the identity, not just the marker. The
  generated skeleton used to ship with every extract() field as None and a
  TODO — a shape, not a contribution — while the wizard had already measured the
  very fields it left blank. It now fills software version, hardware number,
  variant and calibration version from the pattern that found them in the
  contributor's dumps, so the skeleton's match_key agrees with the engine's from
  the first run. The pattern is what travels, never the measured value: a value
  copied in as a literal would match exactly one calibration, which is the same
  mistake the wizard exists to prevent in can_handle(). Whatever the dumps could
  not decide stays a TODO, and the report lists which. The PR description gains
  a Fields measured section, because the measurement a reviewer most needs
  (the ident descriptor carrying variant and calibration version) used to reach
  nobody: it was in the JSON and in none of the files handed over.

• contribute extractor takes as many dumps as you have. The positional
  argument is var...

0.7.7

0.7.7 — 2026-10-17

This file accumulates everything landed since the 0.7.6 release; 0.7.7 is the
next release. It starts with correctness fixes surfaced while reviewing the
remaining items of the 0.7.6 post-release audit, plus a layer/cleanup pass. Two
of the fixes change detection/scan results on real binaries; all of them are
covered by new regression tests. It then adds the synthetic dump generator and
the golden corpus built on it, and the contribution wizard — the tooling half of
this release, which is what a new ECU family now arrives through.

Added

• Contribution wizard (openremap contribute) — the engine that turns "I own
  a dump of an unsupported ECU" into a pull request. The problem it solves is not
  writing the extractor: it is knowing which bytes are stable in that family
  and absent from every family the engine already recognises, which nobody can
  answer by reading their own file. Two verbs:
  • openremap contribute extractor dump.bin --also dump2.bin --brand X --family Y
    measures: who claims the dump today, ranked candidate signatures each tested
    against every known family, collision offsets, and candidate identity fields
    split into constant in the family (belongs in can_handle()) versus
    varies per dump (what a match_key needs). --write additionally emits
    four artifacts: the fixture recipe that goes in the PR instead of the
    dump, an extractor skeleton that already satisfies the review bar
    (measured marker, size gate, exclusion guards harvested from the colliding
    family's own ident token, evidence tags on every return False), the
    fixture-driven test, and a PR report with a Not verified section.
  • openremap contribute check <extractor.py> runs the same bar over an
    existing extractor: it still claims its own fixture, it accepts no other
    family's, a rejected binary leaves no stale evidence, supported_families
    matches what extract() reports, every False path clears the evidence
    (checked by parsing the module, not by pattern-matching text), and the
    extractor is reachable from the registry.
    The wizard executes the skeleton it just rendered against all 89
    known-family fixtures before calling anything verified, and exits non-zero when
    no marker is collision-free or no guard can work — a generated file that has
    never been run is worse than no file. It never writes the contributor's dump
    anywhere, and it refuses a marker that does not occur in their own dumps (that
    would produce a working fixture for a signature the family does not carry).
    Business logic in openremap/core/services/wizard/, thin gates in
    openremap/api/commands/contribute.py and openremap/core/cli/commands/.
    Documented where a reader actually looks: README.md (the Supported ECUs
    section, where somebody checks whether their car is covered, and the
    Contributing section with the two commands) and
    contributing/extractors.md. The known-family fixtures are the repository's own
    _dev/contrib/recipes/ — committed, so anybody with a checkout has them, and
    deliberately not copied into the package. An installed wheel therefore has
    no recipes, and the default refuses with instructions rather than degrading
    silently; --measure-only (the service's measure_only) is the explicit opt-in
    for the half that is pure local computation — who claims the dump, the runs
    stable across the dumps, the family name the dump suggests — with no collision
    check, no recommended marker and no artifacts, and it says all three. It also
    needs neither --brand nor --family: a first look should not require the
    answer it exists to produce (those two name the family the extractor declares,
    so they stay required for the full run). Same
    strict-by-default shape as cook --allow-non-unique. Contributing means opening
    a pull request, which needs a clone anyway; a second copy of the fixtures inside the wheel was tried and
    removed, because it duplicated the knowledge and made the wizard blind to a
    recipe it had itself just written.
• Synthetic dump generator (openremap.core.synth) — builds ECU-like
  binaries from scratch: flash layout, calibration axes, smooth map surfaces,
  a known ground truth, and no byte of any real ROM. Because it is
  deterministic (same parameters + seed → identical bytes), the file can be
  regenerated everywhere instead of shipped: usable as a test fixture, a demo,
  or a reproducer in a bug report — none of which was possible before, since
  real dumps can neither be committed (copyright) nor handed to users.
  Exposed three ways, like every other feature: generate_dump over the API /
  stdio RPC (payload base64) and openremap synth -o file.bin on the CLI.
  Generated files carry no manufacturer fingerprints, so identify reports
  Unknown for them — by design, and pinned as such.
• Golden corpus over synthetic dumps (tests/corpus/) — cases are
  parameter sets in manifest.json, so the corpus is committable (~50 KB for
  three cases) and runs in CI, with no ECU data and no personal data
  involved. Each cases/<id>/golden.json pins: input hashes, planted maps,
  identity, detected map candidates and their scores, the cooked .remap
  recipe (wall-clock fields normalised), the re-patch result — including the
  round-trip property apply(cook(a, b), a) == b — and both refusal layers
  for a tampered recipe (ctx → the patcher refuses with an actionable
  message; ob → strict validation refuses silently; either way nothing is
  written). Regeneration is a reviewed act: _dev/corpus/update_golden.py,
  read the diff, commit it with the change that caused it.

Fixed

• VAG part-number scan no longer stops at an all-digit lookalike. The shared
  scanner (manufacturers/bosch/_vag.py::find_vag_part_number, used by the
  EDC15/EDC16/EDC17/EDC3.x extractors) took the first structurally matching
  run in the dump and applied its "a part number must contain a letter" guard
  afterwards. An all-digit Bosch calibration code of the same shape sitting
  earlier in the file therefore ended the search: the dump reported no part
  number at all even though a valid one existed further on. The scan now walks
  every structural match and returns the first that contains a letter.
  Corpus sweep (1,685 files, before/after): exactly two fields change, both
  gains — two VW Crafter 2.5 TDI EDC16 dumps that reported no OEM part number
  now resolve 074906032AT and 074906032AG (both confirmed by their source
  filenames). No other field moved anywhere in the corpus: no dump lost a part
  number, and no match_key or family changed.
• scan_map_tables no longer serves a stale cached result. The two-stage
  map-scan caches were keyed on (path, mtime, size) alone, so parameters were
  ignored whenever the cache was warm: a second call on the same file with a
  different min_table_score returned the first call's tables (reproduced:
  0.55 → 120 tables, then 0.99 → the same 120 instead of 0). Both caches
  now include every parameter that shapes the result, so an identical request
  still hits the cache while a changed one recomputes.
• Two-stage map scanning decodes HEX/S-Record containers. scan_map_axes
  and scan_map_tables read the file with a raw byte read, bypassing the shared
  loader, so a text Intel HEX / Motorola S-Record input produced axes over ASCII
  text while every other command decoded it first. Both stages now load through
  the API's shared load_binary.
• CONTRIBUTING.md no longer points at a layout that was deleted. Eleven
  references to src/openremap/tuning/manufacturers/… — including the import
  line in the copy-paste example — survived the package re-layout, so the
  public "Adding a New ECU Extractor" guide dead-ended at step 1: the directory
  does not exist and the module does not import. Paths now read
  openremap/core/manufacturers/…. While repairing it, two more gaps in the
  same guide were closed: the worked example now shows the evidence-tag
  contract (_set_evidence() plus tags such as SIZE_MATCH,
  EXCLUSION_CLEAR, FAMILY_STRING) that the confidence scorer depends on and
  which the guide had omitted entirely, and the test-tree snippet no longer
  invents a per-brand sub-directory under tests/tuning/manufacturers/. Two
  further drifts in the same file were corrected: the "Submitting a Pull
  Request" steps said master while the default branch is main.
• CONTRIBUTING.md is now an index, not a textbook. It had grown to 311
  lines of which ~130 were the extractor guide alone — the loudest domain had
  swallowed the page, and every other area of the codebase (checksums,
  decoders, map scoring, instruction scanners, tests) was compressed into a
  table row or missing. It is now an intro plus the binding cross-cutting rules
  (safety notice, setup, code style, PR process, bug reports, binary-file
  policy) and routes to per-domain guides in contributing/:
  extractors.md, checksums.md, decoders.md, map-scoring.md,
  scanners.md and tests.md. Each guide states where the code lives, the
  conventions of that area, and what a PR must demonstrate — which is the
  part that genuinely differs per domain (an extractor must not steal another
  family's claims; a checksum needs family consensus; a decoder needs oracle
  parity; a scoring change must show a corpus before/after).

Changed

• scan_map_tables declares the axis parameters it already read. The
  fallback branch (stage 2 without a prior stage 1) read min_axis_length,
  max_axis_length, min_step and max_step, but none were in the method's
  input_schema — invisible to list_methods consumers and exempt from the
  registry's default/validation handling. All four are now declared, matching
  scan_map_axes.
• RPC list_methods honours kind. The...

0.7.6

0.7.6 — 2026-09-06

This release turns two previously-unidentified lines into full identities and
fixes VAG part-number detection: a new ME7.9.x Bosch extractor, a new Siemens
GS20 (BMW AGS) extractor, and a shared, false-positive-proof VAG part-number
scanner for the EDC15/EDC16/EDC17/EDC3.x extractors. All changes were
validated against the full corpus before and after, with zero regressions.

Added

• ME7.9.x (ME7.9.10) extractor — new Bosch family covering the
  Fiat/Alfa/Lancia 1.4 T-Jet and PSA variants (2006+). These 832KB dumps
  were previously unidentified: their ident block at 0x10000 uses the
  ZZ<< dialect, which the classic ME7 extractor's ZZ<digit> guard
  deliberately skips. The new extractor keys on that dialect plus the
  ME7910/ME7.9 family markers in the calibration area, and extracts the
  10-digit SW number, the 0261…/0261S… HW number and the match key
  (ME7.9.10::<sw>). Registered right after the ME7 extractor — the
  corpus-wide claims sweep shows it claims only the two dumps that were
  previously unclaimed (no detection reordering anywhere else).
• Siemens GS20 extractor — new family covering the 4 previously
  unidentified 64 KB dumps (tests/data/ECUs/Siemens/GS20/GS20_90C0_64KB*).
  Oracle ground truth (ms4x wiki + a community BMW flash-tool corpus): GS20
  is a Siemens BMW automatic-gearbox controller (AGS), HWNR 7544721.
  Detection is a 64 KB
  size gate + the header code GADC9010 + the footer project string
  G2210_0090C0AD10 (both corpus-unique; G2210_ is the oracle's GS20
  ident key). Extracts software_version (7-digit 7557997) and
  calibration_id (GADC9010) with match key GS20::<sw>; hardware_number
  stays None because the oracle HWNR 7544721 is not stored as ASCII in the
  dumps. Field mapping is heuristic — the oracle's GS20 ident numbers are all
  7-digit (HW/ZB/SW), so 7557997 is kept as the version identifier for a
  stable match key (documented in the extractor). A before/after corpus scan
  (1,685 files) shows exactly those 4 files flip from unidentified → GS20 and
  nothing else changes.
• analyze --xref-mode v2 — opt-in code-reference mode whose confirmation
  bonus is score-gated: a table earns the +0.06 only when it is already
  high-confidence (structural score >= 0.85, the corpus-validated
  "calibration, never code" floor) AND referenced by code anywhere in its data
  block. Decoder calibration (load-base detection + arch cascade) stays
  byte-identical to the default v1. Measured on 19 real corpus bins:
  base/arch unchanged (19/19), bonus tightened 396 → 40 with all 40 high-score
  referenced maps kept (median boosted score 0.767 → 0.882). The design
  evolution (v2.0 span-gating → v2.1 first-cell → v2.2 score-gated) is
  documented in core/services/maps/xrefs_v2.py.
• Declarative workflow engine — a small engine (openremap/api/workflows/)
  turns .toml step lists into registered methods with kind="workflow". A
  workflow is a linear, unconditional pipeline: each step is a call to a
  registered method with $input.* / $prev.* / $<as>.* data plumbing, and
  the last step (or an explicit output) is the result. Built-in workflows
  auto-load on import; extra dirs load via load_workflows() or
  OPENREMAP_WORKFLOW_DIRS. Ships identify_cook (identify both binaries →
  cook) as the reference workflow. No code execution — dropping a .toml is
  the third-party extension surface. Malformed or invalid .toml files in
  user-supplied dirs are skipped with a warning (the built-in workflows fail
  fast), so one bad extension file cannot break import openremap.api.

Changed

• diff-maps cell machinery moved to Rust — _read_cells (strided path),
  _best_alignment, and _diff_cells in cli/commands/diff_maps.py are now
  thin wrappers over a new _rs/src/maps/diff_cells.rs module
  (read_cells / best_alignment / diff_cells). The new Rust read_cells
  is a shared primitive that also replaces the duplicated _read_cells copies
  in map_classifier.py and map_exporter.py. The per-cell Python loops are
  gone (~1.2 s of the ~5.9 s diff-maps CLI on the 4 MB pair, est. ~50–100×);
  parity is byte-exact — verified 1:1 against the original pure-Python
  implementations on the real 4 MB tune pair (5,257 read_cells tables incl.
  398 strided, 2,432 best_alignment pairs, 2,432 diff_cells pairs, the
  latter bit-exact including the raw unrounded floats) plus ME7.1.1 and MS43.
• Shared VAG part-number scanner (manufacturers/bosch/_vag.py) — a
  letter-guarded regex used by the EDC15/EDC16/EDC17/EDC3.x extractors'
  OEM part-number resolution. It fixes two real gaps:
  • part numbers with a digit at position 3 (038906016L,
    045906019BQ, 022906032CS) were missed by the old
    0[2-9][A-Z]… pattern;
  • platform-code prefixes (4B0906018DF, 3B0907401) were missed
    entirely.
    The letter-content guard keeps all-digit runs (Bosch HW 0261…/0281…,
    SW 1037…, 9-digit calibration codes) out — a 654-file sweep across
    non-VAG families shows zero false positives, down from the naive
    broadening's 3.
• EDC15 non-Format-D dumps (ALH/AGR etc.) and EDC3.x dumps whose ident
  parsers produced no OEM number now fall back to the shared scanner.
• Denso Subaru family relabel — the petrol ecu_family labels were one
  CPU generation off (confirmed by two independent oracles). Detection is
  unchanged (each size bucket still maps to one clean CPU); only the names
  shifted: SH7055 (160/192 KB) → 68HC16Y5 (Motorola HC16), SH7058
  (512 KB) → SH7055 (SuperH), SH72531 (1 MB) → SH7058 (SuperH). The
  arch table no longer disassembles the 16-bit units as SuperH (capstone has
  no HC16 support), and the health map-count envelopes were re-keyed to the
  new names. Verified on the full corpus before/after — zero claim changes.
• API thin-gate refactor (internal, no output change) — every
  *_binary / *_recipe / *_one / *_image core function moved out of
  openremap/api/commands/ + openremap/api/composite/ into
  openremap/core/services/ (the engine), so the API is now only @register
  handlers + a single shared load_binary helper. The canonical
  to_json_safe() now lives in openremap/core/serialize.py (re-exported by
  api/serialize.py); service functions raise plain ValueError /
  MergeConflict and the gate translates to GuardError. The
  analyze/convert CLI commands now go through the API layer too.

Fixed

• oem_part_number was null for several EDC15/EDC16 dumps that do carry
  a VAG part number (e.g. 038906019CJ, 038906016L) — now extracted and
  matching the external reference labels on the validation set
  (previously 4 misses, now 0).
• Marelli IAW 1AP part number — oem_part_number was always null; the
  extractor now resolves the 10–11 digit part-number run in the code+cal area
  (e.g. 55598889641 at 0x051C0 on the Peugeot 206 dump). Labelled
  heuristic — closed sources report the same value minus the leading 5
  (5598889641).

Removed

• Dead openremap-server JSON-RPC daemon — openremap/server.py (the
  "OpenRemap Studio" daemon, long since superseded) is deleted, and the
  openremap-server console script is replaced by openremap-api, which
  points at the new stdio JSON-RPC transport
  (python -m openremap.api.transport.stdio). The wire format is unchanged
  in shape but errors are now JSON-RPC 2.0 {code, message} objects instead
  of bare strings; every command is dispatched through the openremap.api
  registry (27 methods).

v0.7.5

Full Changelog: v0.7.0...v0.7.5