Data audit: collected but not displayed gaps

[erikdarlingdata]

Summary

Systematic audit of all data collected by both apps vs what's actually shown in the UI. Many columns and even entire tables are collected but never surfaced to users.

Audit performed 2025-02-25. Issue #280 (waiting_tasks / Current Waits view) is tracked separately.

---

High-Impact Gaps

1. memory_grant_stats — rich per-session data mostly hidden

• Lite: 12 columns collected every 1 min, but only SUM(granted_memory_mb) shown as a trend overlay on the Memory chart. No per-session detail grid.
• Dashboard: collect.memory_grant_stats collected but never displayed at all.
• Hidden: session_id, database_name, query_text, requested_memory_mb, used_memory_mb, max_used_memory_mb, ideal_memory_mb, required_memory_mb, wait_time_ms, is_small_grant, dop, query_cost
• Impact: Can't see which sessions have grants, grant efficiency (requested vs used), or whether grants are waiting.

2. file_io_stats — per-file detail collected, only DB-level avg latency displayed

• Lite: GetLatestFileIoStatsAsync() returns per-file data (database, file name, type, physical path, size, delta reads/writes/bytes/stalls) but only used in diagnostic logging. No DataGrid.
• Dashboard: collect.file_io_stats has 18+ columns of per-file detail but only database-level average latency chart is shown.
• Impact: Can't see hot files, individual file stall times, or file sizes.

3. Query stats min/max extremes — both apps only show averages/totals

• Lite query_stats: 15 hidden columns — min/max for physical_reads, rows, grant_kb, used_grant_kb, ideal_grant_kb, reserved_threads, used_threads, spills + total_clr_time, creation_time
• Lite query_store_stats: ~30 hidden columns — min/max for duration, cpu, logical_io_reads/writes, physical_io_reads, clr_time, query_max_used_memory, rowcount, tempdb_space_used, log_bytes_used + execution_type_desc, plan_type, force_failure_count, last_force_failure_reason, compatibility_level
• Dashboard collect.query_store_data: ~14 hidden columns (same pattern — min/max extremes)
• Impact: Users see averages but can't identify worst-case executions or high variance queries.

4. session_wait_stats (Dashboard only) — 12 columns, zero UI

• Collected in collect.session_wait_stats but never displayed anywhere.
• Per-session wait breakdown — different from the aggregate wait_stats trend charts.

---

Medium Gaps

5. No top-waits DataGrid in either app

• Both apps have wait stats trend charts but no grid showing "what are the top waits right now?"
• Lite has WaitStatsRow populated by GetWaitStatsAsync() but no DataGrid in XAML.

6. memory_stats — 4 hidden columns (Lite)

• available_physical_memory_mb, total_page_file_mb, available_page_file_mb, sql_memory_model
• Fetched into MemoryStatsRow but UpdateMemorySummary() only displays 6 of 10 columns.

7. procedure_stats — 8 hidden columns (Lite)

• cached_time, min/max for logical_reads, physical_reads, logical_writes, spills

8. cpu_scheduler_stats — ~13 hidden columns (Dashboard)

• scheduler_id, current_tasks_count, runnable_tasks_count, work_queue_count, pending_disk_io_count, etc.
• Only aggregate chart shown, no per-scheduler detail grid.

9. query_store_stats extra metadata (Lite)

• Beyond min/max extremes: execution_type_desc (Regular/Aborted/Exception), first_execution_time, avg CLR/tempdb/log_bytes, plan_type, force_failure_count, last_force_failure_reason, compatibility_level

---

Not Gaps (Intentional / Redundant)

• HealthParser_Blocking (38 cols) — redundant with blocked_process_reports collection
• HealthParser_Deadlocks (25 cols) — redundant with collect.deadlocks
• HealthParser_SignificantWaits, WaitsByCount, WaitsByDuration — redundant with existing wait stats views
• waiting_tasks — tracked separately in Issue Add Current Waits view to Dashboard and Lite #280

---

Related

• Add Current Waits view to Dashboard and Lite #280 — Current Waits view (waiting_tasks collected in both apps, shown in neither)