Skip to content

Upstreaming

Jump to bottom
Simon Dick edited this page Jun 21, 2026 · 8 revisions

Upstreaming the Amiga port

Working tracker for getting the AmigaOS 3.x port accepted into upstream micropython/micropython. Captures the contributor-guideline requirements, the acceptance blockers/suggestions, and progress against them. The per-phase design log is on Amiga port design; the test runbook is on Amiga port testing. Snapshot date: 2026-06-21.

Legend: ✅ done · ◐ in progress · ☐ not started · ♻ ongoing responsibility

Generative AI policy

Upstream's ContributorGuidelines § Generative AI policy governs human conduct, not the code — there is no code checklist that makes a PR "compliant". What it requires:

  • ♻ The human contributor reads and validates every line before submitting — especially the core py/ changes (asm68k.c/.h, emitn68k.c, the emitnative.c hunks, nlr68k.S).
  • ♻ The human engages personally and technically in review — no copy-pasting between maintainers and a chatbot (translation excepted).
  • ♻ No AI agent opens or drives the PR autonomously; a human submits it and does the review work.

This port is large and heavily AI-assisted, so it will draw extra scrutiny. The bar is genuine understanding: be able to defend each core hunk from your own knowledge, without a chatbot in the loop.

Acceptance blockers / suggestions

Ordered by impact.

# Item Status
1 Split the core py/ native-emitter changes into a separate PR from the port (plan) ◐
2 Drop the modjson.c fork; use upstream extmod/modjson.c ✅
3 Curate the commit history (260 commits → 8, on amiga-port-clean) (plan) ✅
4 Remove the sync_upstream.yml fork-maintenance workflow from the PR ✅
5 Open a pre-submission Discussion + state a long-term maintenance commitment ☐
6 Run codeformat.py (uncrustify 0.71/0.72) and ruff format/check, and say so ✅
7 emit68k.c → emitn68k.c (match emitn<arch>.c naming) ✅
8 Key the native-emitter endianness fixes on MP_ENDIANNESS_BIG, not N_68K ✅
9 Move the 68k NLR implementation into py/ + py.mk (where every other arch lives) ✅
10 CI must build all three variants, not just standard (FPU link paths differ) ☐

Notes on the open items

1 — Split the native emitter. Adding a whole new native-code architecture (asm68k.c/.h ≈ 1k lines, emitn68k.c, emitnative.c hunks, the .mpy ABI enum, NLR) is a large, self-contained review that touches the most sensitive part of the tree. Bundling it with the ~9k-line port makes the combined PR much harder to land. Propose the port first with the native emitter disabled, then the 68k backend as a focused follow-up. The existing #if MICROPY_EMIT_68K / #if N_68K guards make that split trivial.

3 — Commit history. Upstream wants small, logical, clean-rebasing commits. Rebase the 252-commit branch into a coherent set (roughly: core py/ changes; the port; tests; examples; CI; docs).

4 — sync_upstream.yml. ✅ Removed. It was fork-maintenance only. ports_amiga.yml (the port build CI) is retained, but expect it to be adapted to upstream CI conventions and a toolchain story they will host.

5 — Discuss first. A new port is a long-term maintenance commitment for the team. Open a GitHub Discussion (or post on Discord) before the PR, outline the port, and state explicitly who maintains it. A surprise mega-PR for a new architecture is the most likely thing to be closed unread.

10 — All variants in CI. Removing floatconv.c from the FPU builds broke the 68040 link (undefined pow/tgamma) during this work — a standard-only CI would not have caught it. Build standard, 68020fpu, and 68040.

Native-emitter split plan (item #1)

The goal: land the port as a small, reviewable PR with zero core changes, then add the 68k native backend as a focused follow-up. Two stacked PRs.

Why this is clean

Inspection of the branch confirms the split is natural:

  • Every core py/ change on the branch (all 12 files) is native-emitter or NLR related. A bytecode-only port needs no py/ changes at all.
  • The native emitter is gated by one flag, MICROPY_EMIT_68K (ports/amiga/mpconfigport.h). Unset ⇒ bytecode-only.
  • The register NLR (nlr68k) is auto-selected for __m68k__ in py/nlr.h, and its purpose is native try/except (the handler PC rides a register that setjmp's jmp_buf would clobber). With the emitter off, the default setjmp NLR is correct — so the NLR work belongs with the emitter PR.
  • .mpy bytecode load (MICROPY_PERSISTENT_CODE_LOAD) and native .mpy load are independent: the port already sets MICROPY_PERSISTENT_CODE_LOAD_NATIVE = 0, so PR 1 keeps bytecode .mpy load and needs none of the native-arch plumbing.

PR 1 — AmigaOS port (bytecode-only, self-contained)

Diff touches only these trees, no py//extmod//shared/ core edits:

  • ports/amiga/ — all sources except the native config (see below).
  • tests/ports/amiga/ — all smoke tests except the native/viper ones.
  • examples/amiga/, tools/amiga-*, .github/workflows/ports_amiga.yml.

Port config changes vs the current branch:

  • Remove the MICROPY_EMIT_68K enable from mpconfigport.h (defaults off).
  • Do not carry the py/nlr.h __m68k__ block ⇒ NLR falls through to setjmp (correct, just slower on try/except).
  • Soften the README: drop the @micropython.native/viper feature bullet and the native limitations note (they return in PR 2).

Excluded from PR 1 (these are PR 2's): test_native_emitter_smoke.py, test_native_nested_smoke.py, test_native_nlr_smoke.py, test_viper_shift_smoke.py.

PR 2 — 68k native code emitter (core backend), stacked on PR 1

New core files: py/asm68k.c, py/asm68k.h, py/emitn68k.c, py/nlr68k.S, py/nlr68k_jump.c.

Modified core (the audited hunks): py/compile.c (NATIVE_EMITTER case), py/emit.h (method-table decl), py/mpconfig.h (MICROPY_EMIT_68K in the MICROPY_EMIT_NATIVE convenience macro), py/emitnative.c (the N_68K register/codegen blocks + the MP_ENDIANNESS_BIG fixes), py/nlr.h (MICROPY_NLR_M68K + reg count), py/persistentcode.h (MP_NATIVE_ARCH_68K), py/py.mk (asm68k.o, emitn68k.o, nlr68k.o, nlr68k_jump.o).

Port changes: set MICROPY_EMIT_68K = 1; switch NLR to register-based (the py/nlr.h block); restore the README native section; add the four native/viper smoke tests.

Prototype result (de-risked 2026-06-21)

PR 1 was prototyped on a throwaway branch (proto-pr1-no-native): py/ restored to upstream base (all 12 changes dropped) + MICROPY_EMIT_68K=0. Outcome — the "builds without the emitter" assumption holds:

  • All three variants (standard, 68020fpu, 68040) build with zero core py/ changes (git diff of py/ vs base is empty).
  • setjmp NLR works at runtime: try/except, nested try/finally, and re-raise all behave correctly. So PR 1 needs no NLR change.
  • 30/30 non-native port smoke tests pass under vamos, including .mpy bytecode import (which PR 1 keeps). 0 fail, 0 skip.
  • The standard binary is ~25 KB smaller (535,700 vs 560,560 bytes).

Open details (remaining)

  • persistentcode.h may be unneeded even in PR 2. The arch block (incl. MP_NATIVE_ARCH_68K) is guarded by #if MICROPY_PERSISTENT_CODE_LOAD_NATIVE, which the port keeps at 0 (it never saves/loads native .mpy). So base persistentcode.h is correct for PR 1, and PR 2 likely only needs the enum if a future config enables native .mpy. Confirm during PR 2 whether the change is required at all.
  • Optional: the register NLR is also a small perf win for bytecode. It could be its own tiny core PR between PR 1 and PR 2, but keeping PR 1 zero-core-change is the cleaner story.

Build / test matrix

  • PR 1: build standard + 68020fpu + 68040 bytecode-only; run the port smoke suite minus native/viper; spot-check REPL, socket, ssl, asyncio, frozen + .mpy bytecode import.
  • PR 2: build all three with the emitter on; run the native/viper/NLR smoke tests plus the upstream --emit native test variants.

Ordering / prerequisite

This split is best done as part of the history curation (item #3): the current amiga-port (252 commits, based on an old master) needs a rebase onto current upstream master and a squash into logical commits before any PR. Do that rebase first, then carve PR 1 (port, no py/) and PR 2 (py/ backend + enable) off the curated tree. The emitter changes are isolated enough that PR 2's content = the five new py/ files + the listed hunks + the port's native-enable lines; PR 1 = everything else.

History curation plan (item #3)

Scope finding: there is no "rebase", only curation

Measured 2026-06-21: the branch base (5770cede) is the same commit as the real upstream micropython/micropython master — zero upstream commits to catch up to. So "rebase onto master" is currently a no-op for conflicts. The work is entirely history curation of our 260 commits.

(At actual-PR time upstream will have moved; a git rebase upstream/master refresh will be needed then, but the additive nature of the change — +22,225 / −3 lines, almost all new files — makes conflicts unlikely.)

Why an interactive rebase is the wrong tool

The 260-commit history is not worth replaying:

  • 12 merge commits (Merge branch 'micropython:master'...) — linear history is required upstream; these must disappear.
  • ~52 docs/amiga* / phase* commits for files that no longer exist in the tree (the design log moved to this wiki — 0 tracked in HEAD). Pure churn.
  • The test path was renamed mid-history (tests/amiga/ → tests/ports/amiga/); only the final path (35 files) survives.
  • 16 fixup-style commits (fix, again, revert, ...).

Recommended method: squash to fresh history

Rather than rebase -i over 260 commits, rebuild clean history from the final tree (the journey has no upstream value; the destination is +22,225/−3 across 114 files):

  1. git checkout -b amiga-port-clean upstream/master
  2. Lay down the final tree as a small logical commit series (below), using git checkout amiga-port -- <paths> per commit.
  3. Correctness gate: git diff amiga-port amiga-port-clean must be empty — the curated tree is byte-identical to today's branch.
  4. Then carve PR 1 / PR 2 per the split plan.

This drops the merges, the deleted-docs churn, the path rename, and the fixups automatically, and yields linear history by construction.

Result (built 2026-06-21)

amiga-port-clean built from upstream/master — 8 linear commits, 0 merges, and the correctness gate passes (git diff amiga-port amiga-port-clean is empty: byte-identical). The PR1/PR2 boundary is real:

  • PR 1 = commits 1–5 (port, tooling, tests, examples, CI). At the PR1 tip, py/ has zero changes vs upstream and MICROPY_EMIT_68K (0) — i.e. bytecode-only. This content matches the verified proto-pr1-no-native prototype (all three variants build, 30/30 tests).
  • PR 2 = commits 6–8 (asm68k backend; native emitter + register NLR; port enable + emitter tests) — the 12 py/ files, +1347 lines.

Build-tooling consistency fixes are folded into PR 1: the ten amiga-* helper scripts now live under ports/amiga/tools/ (not the cross-port top-level tools/), and CROSS_COMPILE is ?= (overridable) like every other cross port. The relocated build wrapper was re-verified to build.

Branch is local (not pushed). To submit: push amiga-port-clean, open PR 1 from commit 5, then PR 2 stacked on it. Keep the messy amiga-port as an archive until both land.

Proposed logical commits

PR 1 — port (bytecode-only), ~9 commits:

  1. ports/amiga: Add port skeleton (Makefile, config, main, mphal, variants).
  2. ports/amiga: Add AmigaDOS VFS plus os and os.path.
  3. ports/amiga: Add the amiga module (library proxy, intuition, asl, icon, catalog, rexx, volumes).
  4. ports/amiga: Add networking (bsdsocket, select, asyncio).
  5. ports/amiga: Add TLS via AmiSSL.
  6. ports/amiga: Add frozen modules and generated AmigaOS tables.
  7. ports/amiga: Add timing, REPL history, env vars, stdio.
  8. tools: Add Amiga build, codegen, and test tooling.
  9. ports/amiga: Add tests, examples, CI workflow, and README.

PR 2 — native emitter (stacked), ~4 commits:

  1. py/asm68k: Add Motorola 68k assembler backend.
  2. py/emitnative: Add 68k native emitter and big-endian fixes.
  3. py: Add 68k register-based NLR.
  4. ports/amiga: Enable @micropython.native/viper and add emitter tests.

Granularity is tunable; this keeps each commit reviewable and self-contained. The five generated _amiga_*.py tables ship with their generators in commit PR1.6/PR1.8.

Caveats

  • This rewrites history — coordinate before force-pushing amiga-port, or do it on a new branch (amiga-port-clean) and open PRs from that.
  • Keep the messy amiga-port branch as an archive until the PRs land.

Audit findings (no action required)

These were investigated and found acceptable, so they are recorded here to avoid re-litigating them:

  • No other obsolete divergence. modjson.c was the only fork of an upstream file. modsocket.c / modos.c / modtime.c are port-original AmigaOS implementations (parallel to ports/unix/), not diverged copies — keep them.
  • floatconv.c is legitimate — it overrides bebbo gcc 6.5b's broken soft-float libgcc/clib2 routines and provides the pow/tgamma wraps needed on all variants (libm bugs, not soft-float bugs). It is not the modjson pattern; there is no general fix elsewhere.
  • Core py/ diff is clean and well-guarded. 1192 insertions / 3 deletions across 10 files, following the established per-arch backend pattern; every shared-file change is a one-line arch addition or wrapped in #if N_68K, so other ports are provably unaffected. No TODO/FIXME/stub/abort(). The new MP_NATIVE_ARCH_68K enum is appended last (no .mpy ABI renumbering).

Open structural decision

  • NLR file shape. The 68k NLR is a standalone py/nlr68k.S plus a C helper py/nlr68k_jump.c, rather than the single naked-asm .c other arches use (nlrthumb.c-style). This is forced: bebbo m68k gcc ignores __attribute__((naked)), so the register save/restore must be hand-written assembly. nlr68k.S is #if defined(__m68k__)-guarded so it is inert on other ports (verified by building the unix port). Upstream may still prefer a different arrangement — worth raising in the Discussion.

Progress log

Work done so far on this branch (commit hashes are on amiga-port):

  • 3a63676 — Drop modjson fork in favour of extmod/modjson.c. The fork existed only to dodge an mp_obj_t stack-alignment bug that the general MICROPY_OBJ_BASE_ALIGNMENT = aligned(4) fix already solves. Full extmod json suite + port json smoke test pass (14 tests / 286 cases).
  • 888cd94 — Reword modos.c env-var comment to lead with the real-AmigaOS rationale instead of a "Vamos workarounds" heading (comment-only).
  • 19f9f72 — Key big-endian native codegen on MP_ENDIANNESS_BIG (prelude index, generator start-offset, code_state.n_state); the n_state shift is now word-width-sized. Native-generator path verified on 68k.
  • 40ff324 — Rename emit68k.c → emitn68k.c and move the 68k NLR (nlr68k.S, nlr68k_jump.c) into py/ + py.mk. Both amiga variants and the unix port build cleanly; native/viper/NLR smoke tests pass.
  • 301fe33 — Remove sync_upstream.yml fork-maintenance workflow.
  • 8f68627 — Apply codeformat.py + ruff format (uncrustify fixes in py/asm68k.h and main.c; ruff reflow of three examples and a smoke test). codeformat.py is now idempotent on the port.
  • 1163c0d — Fix codespell findings: rename te → tenc in urequests.py, and skip the auto-generated ports/amiga/modules/_amiga_*.py tables (API names like DoubleClick collide with the dictionary).

Linter status (snapshot)

All three repo-wide linters that gate upstream PRs pass on the port:

  • ruff check (lint) — clean.
  • ruff format --check — clean.
  • tools/codeformat.py -c (uncrustify + fixup_c) — clean and idempotent.
  • codespell (2.4.1, CI-style invocation) — clean.

Clone this wiki locally