Skip to content

refactor(api): V2 response envelope, snake_case schema, named views#2729

Open
Innei wants to merge 22 commits into
masterfrom
refactor/v2-api-response
Open

refactor(api): V2 response envelope, snake_case schema, named views#2729
Innei wants to merge 22 commits into
masterfrom
refactor/v2-api-response

Conversation

@Innei
Copy link
Copy Markdown
Member

@Innei Innei commented May 15, 2026

Summary

Breaking 4-phase refactor of the mx-core HTTP response layer per docs/superpowers/specs/2026-05-15-v2-api-response-design.md.

Every successful JSON response is now { data, meta? }; every error is { error: { code, message, details? } }.

Phases

  • Phase 0 — envelope infrastructure. src/common/response/* (envelope/meta/error types, MetaObjectBuilder, ResponseInterceptorV2, AppExceptionFilter, @RawResponse), src/common/views/view.types.ts (parseView), createPagerSchema factory.
  • Phase 1 — snake_case at the schema layer. Drizzle column TS prop names renamed to snake_case across packages/db-schema and rippled through ~32 repositories. The 6 Better Auth tables (readers, accounts, sessions, apiKeys, passkeys, verifications) keep camelCase props for drizzleAdapter compatibility.
  • Phase 2 — per-module migration. All ~45 modules migrated to the { data, meta? } envelope, named *.views.ts Zod views, MetaObjectBuilder, and AppException subclasses.
  • Phase 3 — cleanup. Deleted JSONTransformInterceptor, legacy ResponseInterceptor, translation-entry.interceptor, @TranslateFields, and src/utils/case.util.ts (with its snakeCaseKeys helper); removed the Bypass alias; generic exceptions migrated to AppException; ResponseInterceptorV2 + AppExceptionFilter wired as global APP_INTERCEPTOR / APP_FILTER.

Breaking changes

  • Response envelope. Every JSON endpoint now emits { data, meta? } on success and { error: { code, message, details? } } on failure. Existing V1 consumers must either upgrade to V2-aware client code or pin @mx-space/api-client to the version that ships the legacy response adapter (this PR includes packages/api-client/legacy/response-adapter.ts covering the in-tree consumers).
  • Wire format is snake_case. created_at, is_published, category_id, etc. The Drizzle TS code remains camelCase; ResponseInterceptorV2 converts at the wire boundary.
  • Pager query fields removed. V2 endpoints route through createPagerSchema and drop the following V1 query parameters with no direct replacement:
    • select — use named *.views.ts Zod views (e.g. ?view=card) instead.
    • state — query by domain-specific filters (e.g. ?is_published=…) instead.
    • db_query — removed permanently; raw query passthrough will not return.
    • sortBy / sortOrder — renamed to sort_by / sort_order on the wire.
  • Error codes. Generic plumbing (HTTP_ERROR, VALIDATION_FAILED, RATE_LIMITED, INTERNAL_ERROR) plus per-domain SCREAMING_SNAKE codes via AppErrorCode. Existing numeric ErrorCodeEnum values still flow through BizException and surface as their string names on the wire.

Notable deviations

  • Better Auth tables excluded from the snake_case rename (renaming their Drizzle props breaks drizzleAdapter) — a design-spec gap resolved here.
  • AppErrorCode (new, SCREAMING_SNAKE strings) and legacy ErrorCodeEnum (numeric, used by BizException) coexist by design — generic plumbing moved to AppException, business codes still go through BizException. A follow-up issue can unify them.

Review fixes folded into this PR

  • note.controller.ts /notes/list/:id empty-fallback double-wrap. The no-currentDocument branch returned { data: [], size: 0 }, which ResponseInterceptorV2 wraps to { data: { data: [], size: 0 } } (Symbol-marked envelopes only, by design). Replaced with withMeta([], builder.build()). The contract test for Array.isArray(body.data) would have caught this had the fixture exercised the empty path.
  • Envelope linter widened. scripts/check-controller-response-envelope.ts now flags any controller return literal whose top-level keys include data (previously only { data } and { data, meta } were flagged).
  • CLAUDE.md envelope description corrected. The pass-through rule now reflects the actual Symbol-based detection, and explicitly warns that a literal { data, ... } will be double-wrapped.
  • snakeKey boundary-aware. articleURL -> article_url, HTMLContent -> html_content (was _a_r_t_i_c_l_e__u_r_l). Added a unit test for acronym boundaries.
  • AppExceptionFilter deduped. Triple status >= 500 block factored into logServerError(exception); throttle handling into handleThrottle(ip, url).
  • @BypassCaseTransform JSDoc spells out that the matched subtree is emitted verbatim regardless of depth.
  • PagerDto deprecation expanded to enumerate the V1 fields dropped in V2 (select, state, db_query, sortBy/sortOrder rename).

Verification

208 files changed, +6125 / -3381. Typecheck clean, lint clean, full Vitest suite 1088 passed / 3 skipped / 0 failed.

@safedep
Copy link
Copy Markdown

safedep Bot commented May 15, 2026

⚠️ Scan Failed: Pull Request Too Large

This pull request exceeds GitHub's diff limits and cannot be scanned.

GitHub Limits:

  • Maximum 300 files per diff
  • Maximum 1 MB total diff size

Recommendations:

  • Split this PR into smaller, focused changes
  • Review critical dependency changes manually
  • Contact your team if this is blocking your workflow
    This report is generated by SafeDep Github App

Innei added a commit that referenced this pull request May 20, 2026
Squash PR #2729 into one commit before rebasing onto latest master.
@Innei Innei force-pushed the refactor/v2-api-response branch from da6a054 to 87e6381 Compare May 20, 2026 14:24
Innei added a commit that referenced this pull request May 20, 2026
Squash PR #2729 into one commit before rebasing onto latest master.
@Innei Innei force-pushed the refactor/v2-api-response branch from 87e6381 to fa634ec Compare May 20, 2026 14:39
Innei added a commit that referenced this pull request May 20, 2026
Squash PR #2729 into one commit before rebasing onto latest master.
@Innei Innei force-pushed the refactor/v2-api-response branch from fa634ec to aefbbf1 Compare May 20, 2026 14:52
Innei added a commit that referenced this pull request May 20, 2026
Squash PR #2729 into one commit before rebasing onto latest master.
@Innei Innei force-pushed the refactor/v2-api-response branch from aefbbf1 to 9d42b19 Compare May 20, 2026 16:24
Squash PR #2729 into one commit before rebasing onto latest master.
@Innei Innei force-pushed the refactor/v2-api-response branch from 9d42b19 to 5543aa4 Compare May 20, 2026 17:42
// runs camelcaseKeys on the raw snake_case wire payload before it reaches this
// adapter. So `is_liked` arrives as `isLiked`, `source_lang` as `sourceLang`, etc.

const stripPath = (p: string) => p.split('?')[0].replace(/\/+$/, '') || '/'

const NEW_PORT = process.env.NEW_PORT ?? '2333'
const OLD_PORT = process.env.OLD_PORT ?? '3333'
const filter = process.argv[2] ? new RegExp(process.argv[2], 'i') : null
Innei added 2 commits May 21, 2026 20:39
Previously post/page/note detail handlers only attached translation meta
when the article had actually been translated. V1 consumers expect the
default fields (is_translated, source_lang, available_translations) to
always be present so they can render the language picker / source lang
badge without conditional checks.

Also widen ArticleTranslationSchema.source_lang and target_lang to allow
null so PostgreSQL rows that lack a language column still validate.
The legacy adapter now reconstructs the V1 response shape from V2 raw
payloads on a per-endpoint basis:

- Flattens meta.{translation,interaction,insights,enrichments,related}
  back into items (camelCase, matching default transformResponse output)
- Renames pagination fields (page/totalPages → currentPage/totalPage)
  and synthesizes hasNextPage/hasPrevPage
- Wraps NoteModel detail responses as { data, next, prev } again
- Hoists comment.getByRefId readers/pagination back to the top level
- Strips text/content from aggregate.getTop list items (V2 server omits
  bodies for SSR payload size; V1 didn't, this aligns at the adapter)
- Restores singleFileSizeMb (V2 snake-case emits single_file_size_m_b
  which camelcase upper-cases to singleFileSizeMB)
- Other endpoint-specific tweaks: comment threads, note middle list,
  note topic list, activity presence

Also fix the fetch adaptor losing the Response status getter when
shallow-copying via Object.assign, which caused error handlers to fall
back to status 500 instead of the real HTTP code.

Adds packages/api-client/scripts/smoke-diff.mjs — a parity harness that
hits both ends of a V2/V1 setup with the legacy adapter applied and
deep-diffs the normalized payloads. Used to drive this fix to 0 diffs
across Yohaku's 52-endpoint call surface.

bump @mx-space/api-client to v5.0.2-next.3
@Innei Innei force-pushed the refactor/v2-api-response branch from f7944bc to 9b2363f Compare May 21, 2026 12:39
…ith BasicPagerDto/createPagerSchema

Removes the legacy BizException / BusinessException / ErrorCodeEnum +
CannotFindException / BanInDemoExcpetion / NoContentCanBeModifiedException
dual-track in favor of a single AppErrorCode enum, AppErrorPayloadMap, and
createAppException factory. All 50+ throw sites and 13 test files migrated.
~/common/exceptions/* and ~/constants/error-code.constant.ts are deleted.

Removes the legacy PagerSchema / PagerDto / PagerInput (with V1 select /
state / db_query / camelCase sort fields). The new pager API exposes
BasicPagerSchema/BasicPagerDto/BasicPagerInput for sort-less endpoints
and createPagerSchema(sortKeys) for sortable endpoints; 27 controllers
and schemas migrated. sort_by/sort_order are snake_case on the wire and
typed via z.enum(sortKeys).

Also folds in the earlier ultrareview fixes:
- note.controller.ts /notes/list/:id empty-fallback double-wrap fix
- check-controller-response-envelope.ts now flags any literal whose top
  keys include `data` (catches the bug above)
- CLAUDE.md envelope description corrected (Symbol-based detection)
- snakeKey boundary-aware (articleURL -> article_url, HTMLContent ->
  html_content)
- AppExceptionFilter deduped (logServerError, handleThrottle helpers);
  log/Bark/throttle copy in English
- BypassCaseTransform JSDoc spells out subtree verbatim semantics

All AppErrorCode messages are in English. ENRICHMENT_BROWSER_MODE_REQUIRED
and ENRICHMENT_SCREENSHOT_DISABLED return 409 to match contract tests.

Verification: tsc --noEmit clean, vitest 1163 passed / 0 failed / 3 pending.
@Get('/:type')
@Auth()
async getTypes(@Query() query: PagerDto, @Param() params: FileUploadDto) {
async getTypes(
Innei added 5 commits May 21, 2026 22:53
- file: add CommentUploadsListQueryDto so /comment-uploads/list coerces
  flat ?page=&size= query into numbers; raw @query() previously echoed
  strings into withMeta(...).pagination(...) and tripped ResponseMetaSchema
- markdown: cast meta.slug to string in generateArchive — notes set
  meta.slug to nid (number) and .concat threw a TypeError on export
- say: expose missing GET/:id, POST, PUT/:id, DELETE/:id routes;
  repository already had the methods, controller now wires them so
  admin's manage-says CRUD works
- errors: NOT_FOUND now carries an optional `id` detail to match the
  per-resource error payloads (used by the new say routes)
- api-client: probe both axios-style and ofetch-style envelopes when
  extracting body + $meta; covered by new test
- AggregateController.getSiteMetadata / getAggregateData / getTop now
  accept a CacheableOptions arg ({ next, cache }) so server callers can
  pass Next.js cache hints without dropping to .proxy.* raw path access.
- getAggregateData also accepts `lang` to push the locale into the
  query string (previously consumers had to plumb it through
  ofetch-level langStorage or .proxy.get({ params: { lang } })).
- RequestOptions exposes typed `next?: NextFetchRequestConfig` and
  `cache?: RequestCache` fields, declared structurally so the SDK does
  not pull `next` as a dependency. Adapters that don't recognise them
  (axios, umi) silently drop the keys.
Innei added 5 commits May 22, 2026 01:17
Commit 0d590c2 renamed the legacy adapter's pagination output from the
V2 wire shape (`currentPage` / `totalPage`) to the V3 names
(`page` / `totalPages`) while keeping the commit title as "preserve
flags" — which only described the hasNext/hasPrev change. Downstream
consumers reading the V2 field names (Yohaku's PostPagination,
NoteListPagination, series detail page) silently broke: the pagination
nav rendered empty because `pagination.currentPage` was undefined.

This restores V1 wire parity for the legacy adapter:

- `remapPagination` now emits BOTH key sets: `page`+`currentPage`,
  `totalPages`+`totalPage`. Consumers can read either; the duplicate
  fields cost ~12 bytes per response.
- Adds a dedicated `LegacyPager` / `LegacyPaginateResult` type under
  `packages/api-client/legacy/types.ts` (re-exported from
  `@mx-space/api-client/legacy`). The SDK's main `Pager` interface
  stays clean — when the legacy adapter is eventually deleted, this
  file goes with it and no V2-only fields leak into the V3 type surface.
LegacyPager now declares currentPage/totalPage/hasNextPage/hasPrevPage
as required fields — the legacy adapter unconditionally derives all six
from the V3 envelope's pagination meta, so consumers do not need to
optional-chain them.
Single-file shim (`src/domain/envelope-compat.ts`) that normalizes wire bodies
to the V3 shape before reaching schema decoding, error mapping, and renderer
views. Lets the CLI work against either a V2 or V3 mx-core deployment.

- `normalizeSuccessBody`: lifts V2 root `pagination` under `meta.pagination`.
- `normalizeErrorBody`: unwraps V3 `{ error: { code, message } }` and flattens
  V2 `{ message: string | string[], code? }` to one tuple.
- `mapHttpStatusToError`: prefers V3 SCREAMING_SNAKE codes
  (`<RESOURCE>_NOT_FOUND`, `VALIDATION_FAILED`) over status-derived tags when
  present, so `POST_NOT_FOUND` correctly maps to `ResourceNotFound`.
- `--verbose` prints the detected wire version once per `ApiService`.

View cleanup: `post`/`comment` list renderers now read `meta.pagination` only;
the V2 root-`pagination` read path is dead post-adapter.

Removal is grep-driven via `// COMPAT:envelope` markers and a deletion
checklist in `envelope-compat.ts`'s JSDoc. Design doc:
`packages/cli/docs/specs/2026-05-21-v2-v3-envelope-compat-layer.md`.

Tests: 22 new unit tests for the adapter, V3-fixture branches added to
`cli-post-list` and `cli-error-envelope` integration suites. Full suite:
637/637 passing.
Align response meta with the camelCase-internal / snake_case-wire
convention enforced by `ResponseInterceptorV2`. Snake_case keys
(`total_pages`, `is_translated`, `source_lang`, `target_lang`,
`content_format`, `available_translations`, `is_liked`, `like_count`,
`read_count`, `has_in_locale`) leaked into Zod schemas, the meta
builder, the translation helper, and every controller call site —
forcing manual snake_case in business code and double-converting
through the interceptor.

- meta.types: schemas now declare camelCase keys
- meta-builder: drop `total_pages` alias on LegacyPaginationLike
- helper.translation: buildArticleTranslationMeta emits camelCase
- 12 controllers (note, post, page, draft, reader, webhook, say, file,
  snippet, topic, project, category, aggregate): builder call sites
  switched to camelCase
- unit specs updated; wire shape unchanged (contract specs pass)
- drop dead dep snakecase-keys
Translate all Chinese in comments, logger calls, error messages,
thrown exceptions, Zod descriptions, email subjects/bodies, and
other user-facing strings across apps/core/src.

Preserved (intentional):
- AI prompt example fixtures in modules/ai/ai.prompts.ts
- i18n locale data (LABELS.zh/ja) in mx-space enrichment provider
- Spam-detection dictionaries (block-keywords.json, meaningless-words.json)
- Test fixtures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants