diff --git a/.agent/skills/simplicity-first/SKILL.md b/.agent/skills/simplicity-first/SKILL.md new file mode 100644 index 00000000..a3b5a31b --- /dev/null +++ b/.agent/skills/simplicity-first/SKILL.md @@ -0,0 +1,59 @@ +--- +name: simplicity-first +description: Simplicity and traceability gate for all TransferQueue code written, fixed, or reviewed. Invoke before changing code or reviewing a diff, especially across public APIs, controller and metadata logic, samplers, storage backends, asynchronous execution, or metrics. +--- + +# Put simplicity first + +Keep code readable in one pass. Make each behavior traceable from a public API +or configuration entry through controller, metadata, sampler, and storage +logic to its tests and observability. Treat hidden control flow and unnecessary +indirection as correctness problems, especially across asynchronous or +distributed boundaries. + +1. **Keep execution paths explicit.** Make state transitions, partition and + sample ownership, retry or cleanup behavior, and async boundaries visible at + the point where they matter. Avoid clever control flow that obscures which + component mutates metadata or moves data. + +2. **Use the smallest readable change.** Delete redundancy before adding new + machinery. Do not add configuration, return fields, or extension points for + hypothetical future needs. + +3. **Preserve the control-plane/data-plane boundary.** Keep metadata and + scheduling in the controller layer and payload movement in storage managers, + clients, and backends. Do not route payloads through the controller or bind a + public API to one backend merely to shorten a local implementation. + +4. **Avoid thin abstractions.** Keep one-off logic inline when it remains clear. + Add a helper or type only when it removes real nontrivial duplication, names + an important invariant, or implements an existing interface such as a + sampler, storage manager, or storage client. + +5. **Preserve contracts and capabilities.** Keep public KV, client, and + dataloader behavior; metadata and sampler semantics; supported storage + backends; async and distributed behavior; metrics; and existing defaults. + Simplify implementations without silently narrowing the behavior surface. + +6. **Work from reachable behavior.** Prioritize issues reproducible through + supported APIs, configurations, backends, tutorials, or tests. Still protect + internal invariants required for concurrency, cleanup, and data consistency; + do not invent fixes for states the system cannot enter. + +## Checklist before finishing any change + +- Can a reader trace the change from `transfer_queue.interface`, + `TransferQueueClient`, or `StreamingDataLoader` through the controller and + storage path to a focused test? +- Are control-plane metadata and data-plane payload responsibilities still + separate? +- Are partition isolation, production and consumption status, sampler behavior, + cleanup, and async failure handling preserved where relevant? +- Could this diff be half the size? If unsure, make it smaller. +- Does every new abstraction remove real complexity or implement an existing + TransferQueue interface? +- Does any public API, configuration default, metadata shape, or backend + contract change? If so, verify every affected caller and test. +- Keep comments to concise reasons or invariants; omit run-specific data and + internal paths, while retaining useful upstream issue or PR links. +- One problem = one minimal diff. Do not batch unrelated "improvements". diff --git a/.claude/skills/simplicity-first/SKILL.md b/.claude/skills/simplicity-first/SKILL.md new file mode 120000 index 00000000..7dc09ad2 --- /dev/null +++ b/.claude/skills/simplicity-first/SKILL.md @@ -0,0 +1 @@ +../../../.agent/skills/simplicity-first/SKILL.md \ No newline at end of file diff --git a/.codex/skills/simplicity-first/SKILL.md b/.codex/skills/simplicity-first/SKILL.md new file mode 120000 index 00000000..7dc09ad2 --- /dev/null +++ b/.codex/skills/simplicity-first/SKILL.md @@ -0,0 +1 @@ +../../../.agent/skills/simplicity-first/SKILL.md \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..c508824a --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,36 @@ +# Working rules for AI assistants + +## Code standards (hard rules, not preferences) + +The priority order is explicit: **human readability comes first; coding-agent +traceability is the minimum gate.** A human should understand code in one pass, +and an agent must be able to trace a feature from a public API or configuration +entry through controller, metadata, sampler, and storage logic to observability +and tests without reconstructing hidden control flow. + +1. Over-complex or hard-to-follow code is a **bug**, not a style issue. +2. Reduce complexity and line count — prefer deleting code over adding it. +3. Preserve the control-plane/data-plane boundary: controllers manage metadata + and scheduling; storage managers, clients, and backends move payloads. +4. No over-encapsulation: add an abstraction only for real nontrivial reuse, an + important invariant, or an existing sampler/storage interface. +5. Do NOT remove public APIs, backend support, async/distributed behavior, + metadata semantics, defaults, or observability in the name of simplicity. +6. Prioritize behavior reachable through supported APIs, configurations, + backends, tutorials, or tests; do not fix states the system cannot enter. + +Details: `.claude/skills/simplicity-first` (invoke before any code change). + +## Comments + +Concise "why" only, 2-4 lines, written for an external reader: no job ids, +commit hashes, single-run metrics, or internal cluster paths; keep upstream +issue/PR links. + +## Workflow + +- Reviews report findings only; fixes ship as one minimal diff per issue + after approval. +- Verify with `python -m compileall -q transfer_queue tutorial tests` and + `python -m pytest -q`; use the relevant optional-backend environment for + integration tests and validate shell scripts with `bash -n`. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..43c994c2 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +@AGENTS.md