Skip to content

Benchmark native backend production query shapes - #345

Draft
chubes4 wants to merge 9 commits into
mainfrom
feat/232-production-query-benchmarks
Draft

Benchmark native backend production query shapes#345
chubes4 wants to merge 9 commits into
mainfrom
feat/232-production-query-benchmarks

Conversation

@chubes4

@chubes4 chubes4 commented Aug 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add production-shaped hierarchy and plugin-table inventory workloads
  • attest the configured database backend against the active $wpdb implementation in every decision workload
  • add SQLite and native Homeboy rigs with an immutable installed-site seed for custom-drop-in boot
  • add an identical bounded concurrent HTTP read workload to both rigs
  • validate benchmark rig component-path ownership

Tracks #232.

Current Evidence

The first valid native custom-drop-in run passed end to end:

  • Homeboy run: ca01608d-ad76-47d5-915a-2c7062f218b8
  • reproduction: homeboy bench --path <checkout> --placement local --rig mdi-native --scenario boot-timing --iterations 1 --warmup 0
  • selected workload: exactly boot-timing, source in_tree
  • MDI revision: 34c5d1c
  • active runtime: WP_Markdown_Native_WPDB, enforced by fail-closed workload attestation

The workload had no boot phase/shared state and therefore only proves runtime selection and execution, not a meaningful performance comparison. Prior native timings collected before backend attestation are invalid because WP_SQLite_DB remained active.

The concurrent-read source contract is now present at f4b02ce:

  • both rigs issue 20 host-side requests at concurrency 4 against one runtime
  • every request asserts the exact expected $wpdb class and executes the same bounded wp_users query
  • both rig JSON documents resolve successfully and the native query shape passes tests/smoke-native-order.php
  • runtime comparison remains pending because Homeboy admission refused execution while the controller host was hot and Lab inventory was stale

Upstream Repairs

Remaining Work

  • run concurrent-read against both attested backends after WP Codebox #2415 is available and Homeboy runtime admission recovers
  • run non-skipped production workloads against both attested backends
  • add native correlated NOT EXISTS support before treating wiki-hierarchy as comparable
  • collect repeated comparison evidence only after correctness passes

Verification

  • native runtime activation probe returned WP_Markdown_Native_WPDB and activate_plugin(): ok
  • native boot-timing Homeboy run passed with exact workload provenance
  • php tests/smoke-native-order.php
  • php -l tests/bench-fixtures/concurrent-read-endpoint.php
  • both rig JSON files parse and resolve through homeboy rig show
  • git diff --check

AI Assistance

OpenAI GPT-5.6 Sol via OpenCode traced false backend selection and benchmark orchestration failures across MDI, WP Codebox, and Homeboy Extensions; implemented the fixture, attestation, and concurrent request changes; dispatched upstream repairs; and collected the available evidence under my direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant