Skip to content

perf(ztd-cli): reuse DDL analysis in ztd-config#672

Merged
mk3008 merged 3 commits intomainfrom
codex/664-select-fixture-alias-collision
Mar 25, 2026
Merged

perf(ztd-cli): reuse DDL analysis in ztd-config#672
mk3008 merged 3 commits intomainfrom
codex/664-select-fixture-alias-collision

Conversation

@mk3008
Copy link
Copy Markdown
Owner

@mk3008 mk3008 commented Mar 25, 2026

Summary

ztd-config now reuses shared DDL analysis for linting and table metadata generation, and it skips no-op config writes so telemetry matches the actual persistence path.

What changed

  • Shared DDL analysis is now reused between linting and table metadata assembly in ztd-config.
  • ztd.config.json writes are skipped when the effective config does not change, and telemetry / JSON output now reflect the actual write result.
  • Added coverage for the shared analysis path and the no-op config write path.

Why

  • The goal was to remove redundant parsing / I/O in the ztd-config hot path.
  • On a heavy same-condition fixture, this reduced generate-ztd-config from about 175 seconds to about 89 seconds.
  • The remaining cost is still concentrated in generate-ztd-config; config writes are not the bottleneck.

Verification

  • pnpm --filter @rawsql-ts/ztd-cli build
  • pnpm --filter @rawsql-ts/ztd-cli test -- ztdProjectConfig.unit.test.ts ztdConfigCommand.telemetry.unit.test.ts ztdConfig.unit.test.ts
  • pnpm --filter @rawsql-ts/testkit-core test -- ddlLint.test.ts
  • Heavy same-condition benchmark on 1000 tables, 120 columns/table, 10 indexes/table, 1 file:
    • current: generate-ztd-config avg 89,254.501ms
    • baseline: generate-ztd-config avg 174,966.294ms

Notes

  • The commit was created after a pre-commit hook failure in unrelated workspace tests. The failure was not judged to be caused by this change set itself.
  • A changeset was added for @rawsql-ts/ztd-cli and @rawsql-ts/testkit-core.

Summary by CodeRabbit

  • New Features

    • Exposes a detailed DDL analysis result alongside diagnostics for reuse across workflows.
  • Chores

    • Consolidated DDL processing so linting and metadata generation share a single analysis pass.
    • Skips no-op configuration writes so telemetry and persisted state reflect actual changes.
  • Tests

    • Added unit tests covering DDL analysis behavior, config write-skipping, and telemetry reporting.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Mar 25, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1130e80d-6b0f-4f0c-a1d2-1a35a6ae050c

📥 Commits

Reviewing files that changed from the base of the PR and between 68b385e and 56aa259.

📒 Files selected for processing (1)
  • packages/testkit-core/tests/ddlLint.test.ts
✅ Files skipped from review due to trivial changes (1)
  • packages/testkit-core/tests/ddlLint.test.ts

📝 Walkthrough

Walkthrough

This PR extracts a shared DDL analysis (parsing + linting) returned as DdlSourceAnalysis, refactors ztd-config to reuse that analysis for diagnostics and table metadata, and makes project-config persistence skip no-op writes (so telemetry reflects only actual persisted changes).

Changes

Cohort / File(s) Summary
Changeset Entry
.changeset/soft-bears-breathe.md
Adds patch release note for @rawsql-ts/ztd-cli and @rawsql-ts/testkit-core.
DDL Analysis Extraction
packages/testkit-core/src/fixtures/ddlLint.ts, packages/testkit-core/src/index.ts
Adds exported DdlSourceAnalysis type and analyzeDdlSources(...); refactors lintDdlSources to delegate and return diagnostics only.
DDL Linting Tests
packages/testkit-core/tests/ddlLint.test.ts
New test validating statement grouping and parity between analyzeDdlSources diagnostics and lintDdlSources.
ztd-config Pipeline Refactoring
packages/ztd-cli/src/commands/ztdConfig.ts
Uses analyzeDdlSources result for both linting and metadata; adds collectCreateTableStatements and buildTableMetadataFromCreateStatements.
Config Command Persistence
packages/ztd-cli/src/commands/ztdConfigCommand.ts
Tracks configUpdated from actual write result and gates diagnostic emission/telemetry on it.
Project Config Write Logic
packages/ztd-cli/src/utils/ztdProjectConfig.ts
writeZtdProjectConfig now accepts baseConfig, returns boolean indicating whether a file write occurred, compares serialized configs and skips no-op writes.
Config Telemetry & Persistence Tests
packages/ztd-cli/tests/ztdConfigCommand.telemetry.unit.test.ts, packages/ztd-cli/tests/ztdProjectConfig.unit.test.ts
Adds tests asserting telemetry shows configUpdated: false when no write occurs and that writeZtdProjectConfig skips rewriting unchanged files (mtime unchanged).

Sequence Diagram

sequenceDiagram
    participant CLI as ztd-config CLI
    participant Analysis as DDL Analysis
    participant Linting as Linting
    participant Metadata as Table Metadata
    participant Config as Config Persist

    CLI->>Analysis: analyzeDdlSources(sources)
    activate Analysis
    Analysis->>Analysis: parse SQL & produce AST groups + diagnostics
    Analysis-->>CLI: return analysis (create/alter/index + diagnostics)
    deactivate Analysis

    CLI->>Linting: validate using analysis.diagnostics
    Linting-->>CLI: diagnostics result

    CLI->>Metadata: buildTableMetadataFromCreateStatements(analysis.createStatements)
    activate Metadata
    Metadata->>Metadata: construct TableMetadata (dedupe & normalize)
    Metadata-->>CLI: table metadata
    deactivate Metadata

    CLI->>Config: writeZtdProjectConfig(rootDir, overrides, baseConfig)
    activate Config
    Config->>Config: compare serialized base vs final config
    alt configs match
        Config-->>CLI: return false
    else configs differ
        Config->>Config: write file
        Config-->>CLI: return true
    end
    deactivate Config

    CLI->>CLI: emit telemetry only if configUpdated == true
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I hopped through schemas, parsed each line,
Shared the findings, linted fine.
No needless write, the hay stays dry,
Telemetry truth beneath the sky.

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'perf(ztd-cli): reuse DDL analysis in ztd-config' accurately describes the main performance optimization: reusing a shared DDL analysis instance instead of analyzing DDL separately for linting and table metadata generation.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/664-select-fixture-alias-collision

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
packages/testkit-core/tests/ddlLint.test.ts (1)

27-30: Consider asserting alterStatements and indexStatements for complete coverage.

The test only checks createStatements.length. Adding assertions for the other statement arrays would verify the full analysis result:

♻️ Optional enhancement
   const analysis = analyzeDdlSources(sources);

   expect(analysis.createStatements).toHaveLength(1);
+  expect(analysis.indexStatements).toHaveLength(1);
+  expect(analysis.alterStatements).toHaveLength(1);
   expect(analysis.diagnostics).toEqual(lintDdlSources(sources));
 });
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/testkit-core/tests/ddlLint.test.ts` around lines 27 - 30, The test
currently only asserts analysis.createStatements length and diagnostics
equality; extend it to also assert analysis.alterStatements and
analysis.indexStatements (e.g., toHaveLength(expected) and/or toEqual(expected
arrays) depending on test data) to fully cover the analyzeDdlSources result and
ensure consistency with lintDdlSources; reference the analyzeDdlSources return
object (createStatements, alterStatements, indexStatements, diagnostics) and the
lintDdlSources(sources) comparison when adding these assertions.
packages/ztd-cli/src/commands/ztdConfigCommand.ts (1)

311-320: Minor: Watcher cleanup references wrong handler.

The watcher registers scheduleReloadIfDdl but cleanup removes scheduleReload. This is pre-existing code (not part of this PR), but the mismatch means listeners aren't properly removed on shutdown.

♻️ Suggested fix (outside PR scope)
-      watcher.off('add', scheduleReload);
-      watcher.off('change', scheduleReload);
-      watcher.off('unlink', scheduleReload);
+      watcher.off('add', scheduleReloadIfDdl);
+      watcher.off('change', scheduleReloadIfDdl);
+      watcher.off('unlink', scheduleReloadIfDdl);
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/ztd-cli/src/commands/ztdConfigCommand.ts` around lines 311 - 320,
The watcher registers listeners using scheduleReloadIfDdl but the shutdown
cleanup in stop() removes scheduleReload, so listeners are not removed; update
the cleanup to call watcher.off('add', scheduleReloadIfDdl),
watcher.off('change', scheduleReloadIfDdl), and watcher.off('unlink',
scheduleReloadIfDdl) (replace references to scheduleReload with
scheduleReloadIfDdl) inside the stop function to match the registered handlers
and ensure proper teardown.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/ztd-cli/src/utils/ztdProjectConfig.ts`:
- Around line 125-142: The comparison uses finalSerialized before the connection
merge, so changes to connection are ignored; to fix, call
mergeConnectionConfig(baseConfig.connection, overrides.connection) and apply its
result to finalConfig (or delete finalConfig.connection) immediately after
creating finalConfig from mergeProjectConfig (i.e., just after const finalConfig
= mergeProjectConfig(baseConfig, overrides)) and only then compute
baseSerialized and finalSerialized for the existingPath/no-op check; reference
mergeProjectConfig, mergeConnectionConfig, finalConfig, baseConfig, overrides,
resolveZtdConfigPath and writeFileSync to locate the code to reorder.

---

Nitpick comments:
In `@packages/testkit-core/tests/ddlLint.test.ts`:
- Around line 27-30: The test currently only asserts analysis.createStatements
length and diagnostics equality; extend it to also assert
analysis.alterStatements and analysis.indexStatements (e.g.,
toHaveLength(expected) and/or toEqual(expected arrays) depending on test data)
to fully cover the analyzeDdlSources result and ensure consistency with
lintDdlSources; reference the analyzeDdlSources return object (createStatements,
alterStatements, indexStatements, diagnostics) and the lintDdlSources(sources)
comparison when adding these assertions.

In `@packages/ztd-cli/src/commands/ztdConfigCommand.ts`:
- Around line 311-320: The watcher registers listeners using scheduleReloadIfDdl
but the shutdown cleanup in stop() removes scheduleReload, so listeners are not
removed; update the cleanup to call watcher.off('add', scheduleReloadIfDdl),
watcher.off('change', scheduleReloadIfDdl), and watcher.off('unlink',
scheduleReloadIfDdl) (replace references to scheduleReload with
scheduleReloadIfDdl) inside the stop function to match the registered handlers
and ensure proper teardown.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7cadef4a-a3a4-4a1f-aa33-e4d8880602e4

📥 Commits

Reviewing files that changed from the base of the PR and between 501437a and 68b385e.

📒 Files selected for processing (9)
  • .changeset/soft-bears-breathe.md
  • packages/testkit-core/src/fixtures/ddlLint.ts
  • packages/testkit-core/src/index.ts
  • packages/testkit-core/tests/ddlLint.test.ts
  • packages/ztd-cli/src/commands/ztdConfig.ts
  • packages/ztd-cli/src/commands/ztdConfigCommand.ts
  • packages/ztd-cli/src/utils/ztdProjectConfig.ts
  • packages/ztd-cli/tests/ztdConfigCommand.telemetry.unit.test.ts
  • packages/ztd-cli/tests/ztdProjectConfig.unit.test.ts

Comment on lines +125 to +142
const finalConfig = mergeProjectConfig(baseConfig, overrides);
const existingPath = resolveZtdConfigPath(rootDir);
const existingConfigPresent = existsSync(existingPath);
const baseSerialized = `${JSON.stringify(baseConfig, null, 2)}\n`;
const finalSerialized = `${JSON.stringify(finalConfig, null, 2)}\n`;
if (existingConfigPresent && baseSerialized === finalSerialized) {
return false;
}

const resolvedConnection = mergeConnectionConfig(baseConfig.connection, overrides.connection);
if (resolvedConnection) {
finalConfig.connection = resolvedConnection;
} else {
delete finalConfig.connection;
}

const serialized = `${JSON.stringify(finalConfig, null, 2)}\n`;
writeFileSync(resolveZtdConfigPath(rootDir), serialized, 'utf8');
writeFileSync(existingPath, `${JSON.stringify(finalConfig, null, 2)}\n`, 'utf8');
return true;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Connection config is merged after the no-op comparison, causing incorrect change detection.

The finalConfig used in finalSerialized (line 129) doesn't include the merged connection from lines 134-139. This means:

  1. If baseConfig.connection differs from overrides.connection, the serialized comparison won't detect the change
  2. The function may return false (no write) while the actual written content differs from baseSerialized

Move the connection merge before the serialization/comparison:

🐛 Proposed fix
 export function writeZtdProjectConfig(
   rootDir: string,
   overrides: Partial<ZtdProjectConfig> = {},
   baseConfig: ZtdProjectConfig = loadZtdProjectConfig(rootDir)
 ): boolean {
-  const finalConfig = mergeProjectConfig(baseConfig, overrides);
+  let finalConfig = mergeProjectConfig(baseConfig, overrides);
+
+  const resolvedConnection = mergeConnectionConfig(baseConfig.connection, overrides.connection);
+  if (resolvedConnection) {
+    finalConfig = { ...finalConfig, connection: resolvedConnection };
+  } else {
+    const { connection: _, ...rest } = finalConfig;
+    finalConfig = rest as ZtdProjectConfig;
+  }
+
   const existingPath = resolveZtdConfigPath(rootDir);
   const existingConfigPresent = existsSync(existingPath);
   const baseSerialized = `${JSON.stringify(baseConfig, null, 2)}\n`;
   const finalSerialized = `${JSON.stringify(finalConfig, null, 2)}\n`;
   if (existingConfigPresent && baseSerialized === finalSerialized) {
     return false;
   }

-  const resolvedConnection = mergeConnectionConfig(baseConfig.connection, overrides.connection);
-  if (resolvedConnection) {
-    finalConfig.connection = resolvedConnection;
-  } else {
-    delete finalConfig.connection;
-  }
-
   writeFileSync(existingPath, `${JSON.stringify(finalConfig, null, 2)}\n`, 'utf8');
   return true;
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/ztd-cli/src/utils/ztdProjectConfig.ts` around lines 125 - 142, The
comparison uses finalSerialized before the connection merge, so changes to
connection are ignored; to fix, call
mergeConnectionConfig(baseConfig.connection, overrides.connection) and apply its
result to finalConfig (or delete finalConfig.connection) immediately after
creating finalConfig from mergeProjectConfig (i.e., just after const finalConfig
= mergeProjectConfig(baseConfig, overrides)) and only then compute
baseSerialized and finalSerialized for the existingPath/no-op check; reference
mergeProjectConfig, mergeConnectionConfig, finalConfig, baseConfig, overrides,
resolveZtdConfigPath and writeFileSync to locate the code to reorder.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant