feat: performance profiling harness for native adapter templates

[mickdarling]

Summary

Build an automated performance profiling harness that tests JXA templates at multiple scales against a live macOS application, producing structured performance metadata for each operation.

Context

During Apple Mail adapter development, we discovered that JXA access patterns have wildly different performance characteristics that cannot be predicted from the sdef or documentation:

Pattern	3 items	30 items	1,600 items	112k items
Index iterator	~1s	~3s	~15s	works (with limit)
Batch property read	~1s	~3s	TIMEOUT	TIMEOUT
whose filter	~2s	~5s	TIMEOUT	TIMEOUT
Full enumeration	~1s	~3s	TIMEOUT	TIMEOUT

These cliffs are invisible until you test at realistic scale. The profiling harness makes this testing systematic and automated.

What to Build

Profiler Module (src/template-profiler.ts)

Input: A template (JXA script with params) + target app + test parameters at multiple scales
Output: Performance report with scaling characteristics

Profiling Process

For each template:

1. Warm-up run — execute once to ensure Mail.app is responsive
2. Scale 1 — run with limit=1, measure latency
3. Scale 5 — run with limit=5, measure latency
4. Scale 10 — run with limit=10, measure latency
5. Scale 50 — run with limit=50, measure latency (with 30s timeout)
6. Scale 100 — run with limit=100, measure latency (with 60s timeout)
7. If any scale times out, record the failure threshold

Output Format

{
  "operation": "list_messages",
  "profiled_at": "2026-04-04T15:00:00Z",
  "target_app": "Mail.app 16.0",
  "target_os": "macOS 15.4",
  "mailbox": "DollHouseMCP (1,603 messages)",
  "results": [
    {"scale": 1, "latency_ms": 850, "status": "ok"},
    {"scale": 5, "latency_ms": 2100, "status": "ok"},
    {"scale": 10, "latency_ms": 4200, "status": "ok"},
    {"scale": 50, "latency_ms": 18500, "status": "ok"},
    {"scale": 100, "latency_ms": null, "status": "timeout", "timeout_ms": 60000}
  ],
  "recommendation": {
    "max_safe_limit": 50,
    "latency_per_item_ms": 380,
    "pattern_viable": true
  },
  "eliminated_alternatives": [
    {"pattern": "batch_property_read", "failed_at_scale": 50, "reason": "AppleEvent timeout"},
    {"pattern": "whose_filter", "failed_at_scale": 1, "reason": "AppleEvent timeout on 1600-message mailbox"}
  ]
}

CLI

mcpaql-profile-template --template templates.json --operation list_messages \
  --app Mail --account Google --mailbox DollHouseMCP \
  --scales 1,5,10,50,100 --timeout 60000

Or profile all templates:

mcpaql-profile-template --template templates.json --all \
  --app Mail --account Google --mailbox DollHouseMCP

Integration with Pipeline

The profiler runs as part of Stage 3 (Template Research):

1. Auto-generate candidate templates (from sdef)
2. Profile each candidate ← this tool
3. Eliminate non-viable patterns
4. Select winning patterns
5. Output final templates.json with performance metadata

Acceptance Criteria

• Profiles a single template at configurable scales
• Detects timeouts and records failure threshold
• Produces structured JSON performance report
• CLI interface for single-operation and batch profiling
• Performance metadata format matches proposed performance field in template overrides
• Tested against Apple Mail templates (validates known results)

References

• Performance findings: spec: Define template research and authoring stage for native adapter pipeline spec#238 (comments, 2026-04-04)
• Caching strategy: spec: Define adapter-level caching and pre-fetch strategy for slow transports spec#239
• Template research stage: spec: Define template research and authoring stage for native adapter pipeline spec#238