Skip to content

fix(#805): reduce DEBUG log noise from binlog reader#806

Merged
morgo merged 3 commits into
block:mainfrom
morgo:reduce-debug-log-noise
May 2, 2026
Merged

fix(#805): reduce DEBUG log noise from binlog reader#806
morgo merged 3 commits into
block:mainfrom
morgo:reduce-debug-log-noise

Conversation

@morgo
Copy link
Copy Markdown
Collaborator

@morgo morgo commented May 2, 2026

Summary

Trims two large sources of noise from CI DEBUG logs (the bulk of the volume in #805) and replaces aggregate signal lost from the trimming with a single bookend log per watermark-optimization phase.

  • pkg/repl/client.go — enumerate the five binlog stream-housekeeping events (GTIDEvent, TableMapEvent, XIDEvent, FormatDescriptionEvent, PreviousGTIDsEvent) as explicit empty cases in the type switch. They fall through silently rather than tripping the "Received unknown event type" log (~68k lines on the sample run). The default case still logs, so a future row-event variant we don't recognize will not be silently dropped.
  • pkg/repl/client.go — drop the per-rotate Debug log (~4.5k lines on the sample run). The rotate handler logic is unchanged.
  • pkg/repl/subscription_{map,buffered,queue}.go — add atomic counters (keysAdded, keysDroppedAbove, keysSkippedBelow) and emit a single Info-level \"watermark optimization toggled\" log on each SetWatermarkOptimization call with table, enabled, all three counters, and current delta_len. Counters are swapped to zero on the toggle so each phase reports its own totals. The existing per-row Debug logs are kept for zoom-in debugging — the bookend log is what makes them grep-skippable rather than essential.
  • pkg/repl/subscription_{map,buffered}_test.go — three existing tests now assert the path taken via the new counters in addition to the post-state:
    • TestKeyAboveWatermark
    • TestFlushUnderLockBypassesWatermark
    • TestBufferedMapFlushWithoutLockRespectsWatermark

Closes #805. Companion to the diagnostic logging in #750 for #746 — makes those logs much easier to read on the next CI hit.

Test plan

  • go build ./... clean
  • go vet ./... clean
  • go test ./pkg/repl/ -count=1 green locally against MySQL 8.0
  • CI green
  • Spot-check a fresh CI DEBUG log to confirm the bookend log appears at expected transitions and the volume drop is in line with the estimate (~38% from the trimming alone)

🤖 Generated with Claude Code

morgo added 3 commits May 2, 2026 07:27
Trim two large sources of noise that dominate CI DEBUG logs without
adding diagnostic value, and add per-phase summary logs around the
watermark optimization so the remaining per-row Debug lines become
grep-skippable rather than essential.

- pkg/repl/client.go: list the five known stream-housekeeping events
  (GTID, TableMap, XID, FormatDescription, PreviousGTIDs) explicitly
  in the type switch so they fall through silently. Default case still
  logs so an unrecognized future row-event variant is not silently
  dropped. Also drop the per-rotate Debug log.

- pkg/repl/subscription_{map,buffered,queue}.go: add atomic counters
  (keysAdded, keysDroppedAbove, keysSkippedBelow) and emit a single
  Info-level "watermark optimization toggled" log on each transition
  with the per-phase totals plus delta_len. Counters reset on toggle.
  Per-row Debug logs are kept for zoomed-in debugging.

- pkg/repl/subscription_{map,buffered}_test.go: tighten three existing
  tests to assert the path taken via the new counters in addition to
  the post-state.
@morgo morgo enabled auto-merge May 2, 2026 13:38
@morgo morgo merged commit c4466a1 into block:main May 2, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DEBUG logs are too verbose to navigate when triaging flakes

2 participants