Enhancement: Improve tracing performance

[hhubert6]

I've tried several approaches to improve performance, but none of them yielded a significant enough impact to comfortably handle the MemoryExplosion examples included in the dev app.

Some of the things I tried that give noticeable difference:

• Using file port in :dbg.tracer with :dbg.trace_client
  • :follow_file creates challenges of manual file size management (I haven't gone deeper on that)
  • :file with FileWrapSpec manages file size well but does not support streaming traces live so it becomes really problematic with iterative traces fetching
• Using ip port in :dbg.tracer with :dbg.trace_client
• Replacing :dbg.tracer with plain :erlang.trace
• Using custom erl_tracer NIF for reducing trace size by dropping some field like socket from assigns in render/1 callback or assigns from socket in other callback (e.g. handle_event/3)
• Dynamic spawning of TraceHandler GenServer for load balancing of incoming traces

All listed options give somewhat noticeable difference in performance and/or memory usage but each potentially creates known challenges or ones that may occur later. That's why I went with simple replacement of :dbg.tracer with :erlang.trace as it's least prone to unknown issues, the easiest to manage and gives visible increase in performance.

Based on what I've researched, attempting to improve performance beyond that would require fundamental and extensive redesign of the entire tracing logic, likely involving different design assumptions than those we initially made.

[Copilot]

Pull request overview

This PR refactors the tracing layer to reduce runtime overhead by replacing :dbg usage with direct :erlang.trace/3 and :erlang.trace_pattern/3 calls, and updates callback tracer docs/tests accordingly.

Changes:

• Replace :dbg-backed tracer with a dedicated registered tracer process and direct BEAM tracing BIFs (LiveDebugger.API.System.Dbg).
• Adjust tracing flag usage to reduce global tracing overhead and enable per-LiveView process tracing where needed.
• Update documentation/comments and test expectations to match the new tracing behavior.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/services/callback_tracer/gen_servers/tracing_manager_test.exs	Updates mock expectations for new per-process tracing flags.
test/services/callback_tracer/actions/tracing_test.exs	Updates mock expectations for revised Dbg.process/1,2 flag lists.
lib/live_debugger/structs/trace.ex	Clarifies trace id documentation.
lib/live_debugger/services/callback_tracer/README.md	Updates tracer architecture documentation away from :dbg.tracer.
lib/live_debugger/services/callback_tracer/process/tracer.ex	Updates docs/comments to reflect the new tracer handler model.
lib/live_debugger/services/callback_tracer/gen_servers/tracing_manager.ex	Broadens :DOWN handling to match any tracer termination reason.
lib/live_debugger/services/callback_tracer/gen_servers/trace_handler.ex	Updates comments to reference the new Dbg-started tracer process.
lib/live_debugger/services/callback_tracer/actions/tracing.ex	Adjusts global and per-process tracing flag configuration for performance.
lib/live_debugger/api/system/dbg.ex	Core refactor: implements tracing via :erlang.trace/3 and :erlang.trace_pattern/3 with a custom tracer process.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[kraleppa]

Very nice!

[kraleppa]

nice 🐬

[GuzekAlan]

LGTM 🦈