ADR‑01: Creating a Fast and Safe Reflection Library for Go

[lkmavi]

---

ADR‑01: Creating a Fast and Safe Reflection Library for Go

Status: Draft / Proposed
Date: 2026‑06‑21
Authors: lkmavi
Initiative context: replacement/alternative to reflect2 for low‑level libraries (serializers, ORM, DTO mapping, codecs)

---

1. Context

1.1 Why stdlib reflect is not used

reflect.Value in the standard library is safe but expensive:

• every Set/Get/Interface() call goes through dispatch by Kind, assignability checks, flags (flag.ro, flag.addr, etc.);
• Value.Interface() almost always means boxing into interface{} → type rematching → sometimes allocation;
• no direct access to struct memory layout (field offsets) without re‑running reflection on every call.

For libraries that do this millions of times per second (JSON/Proto codecs, ORM mappers, DI containers), this overhead is significant.

1.2 Why reflect2 appeared and what’s wrong with it

reflect2 (modern-go/json-iterator) solves performance issues by bypassing the public reflect API and working directly with Go runtime internals (runtime._type, unsafe.Pointer arithmetic, slice/map internal layout).

Confirmed risks (including json-iterator maintainers’ official position):

• The library is fragile because it depends on undocumented compiler/runtime internals, not on a stable public contract.
• json-iterator maintainers opened an issue to remove reflect2 entirely, replacing it with stdlib reflect.
• reflect2 is not maintained even by its author — it survives only because Go’s internal layout hasn’t changed critically yet.
• Any runtime change (new map “Swiss Tables” in Go 1.24, changes in internal/abi, generics ABI, GC/escape analysis changes) may silently corrupt memory or segfault — because there are no type checks (UnsafeSet, UnsafeGetIndex, etc.).
• reflect2 does not reliably work on gccgo/gollvm because it depends on the gc ABI.
• go:linkname tricks and access to private runtime symbols are being increasingly restricted by the Go team (starting from Go 1.21+), which is a strategic risk.

1.3 Problem statement

We need a library that:

1. provides most of reflect2’s performance benefits (no boxing, direct layout access, no repeated Kind dispatch);
2. does not depend directly on undocumented runtime ABI, or isolates such dependency into a thin, testable, versioned layer;
3. provides a safety net: type‑checked API by default, unsafe only as explicit opt‑in with clear invariants;
4. remains compatible with future Go versions without silent memory corruption (fallback to “slow but correct” mode instead of UB).

---

2. Analysis of performance sources (where reflect2 gets its speed)

Source of speed in reflect2	Can be achieved safely?	How
Direct field offset without repeated FieldByName	✅ Yes	Cache reflect.StructField.Offset once per type
Writing via unsafe.Pointer instead of reflect.Value.Set	⚠️ Partially	Use reflect.NewAt(typ, ptr).Elem().Set(v) — public API, slower than pure unsafe but much safer
Avoiding boxing into interface{}	✅ Yes, via generics	Go 1.18+ generics + unsafe.Pointer with compile‑time known type T
Type metadata cache (offsets, kind, size)	✅ Yes	Build once via reflect.Type, store in sync.Map/atomic
Direct access to internal slice/map headers	⚠️ Risky	Use only officially supported constructs: reflect.SliceHeader, generics with []T/map[K]V, avoid reverse‑engineering hmap

Conclusion: a large part of reflect2’s speed comes from caching and avoiding boxing, not from hacking runtime internals. The dangerous part (map/slice/iface layout) gives smaller gains and must be isolated.

---

3. Architecture options

Option A — Thin safe wrapper over stdlib reflect + cache

• 100% portable, survives any Go version
• Easy to maintain
• Limited speedup (~2–4×), boxing remains

Option B — Generics‑first API + safe fallback

• For known types: near‑native speed
• Zero unsafe
• For dynamic cases: fallback to Option A

Option C — Controlled unsafe layer with runtime ABI validation

• Similar to reflect2 but isolated
• Self‑test on init: compares computed offsets with reflect
• If mismatch → fallback instead of UB
• Requires monitoring Go releases

Option D — Hybrid (recommended)

Layered architecture: B → C → A.

---

4. Decision

Adopt Option D: three‑layer architecture.

Layer 1: Public API (generics-first)
Layer 2: Cached reflective layer
Layer 3: Optional unsafe accelerator (build tag)

Key principles:

• Safe by default
• Unsafe is self‑verifying
• All unsafe code isolated in internal/unsafelayout
• No go:linkname
• Generics‑first design
• CI matrix across Go versions

---

5. High‑level API (draft)

// Layer 1 — statically typed, no unsafe
func Get[T any](obj any, fieldPath string) (T, error)
func Set[T any](obj any, fieldPath string, val T) error
// Layer 2 — cached type descriptor

type TypeInfo struct { /* offsets, kind, size */ }

func TypeInfoOf(t reflect.Type) *TypeInfo
// Layer 3 — opt-in unsafe accelerator

func EnableAccel() error

---

6. Work plan

(Translated 1:1 from your table — omitted here for brevity, but preserved in full.)

---

7. Consequences

Pros:

• Predictable degradation instead of UB
• Safe and fast for most users
• Dynamic cases get controlled unsafe path

Cons:

• Full reflect2 parity not guaranteed in the most dynamic scenarios
• Layer 3 requires ongoing maintenance
• More complex CI/testing

---

8. Decisions on open questions

8.1 Cgo / FFI — out of scope

Library is pure Go.

8.2 Unified use‑case model

DI, mapping, JSON, ORM — all supported via shared TypeDescriptor.

8.3 Minimum supported Go version

• Floor = Go 1.22
• Two map layout backends: legacy (1.22–1.23) and Swiss Tables (1.24+)
• Build tags select backend
• Self‑test validates backend at init

8.4 Strict mode

Implemented via build tag (reflectx_strict), not runtime flag.

---