Tracing roadmap: where does mode-1 (EVENT_PC) belong in mlir-aie?

[FIM43-Redeye]

Hi! Picking up the thread from #3176, @hunhoffe suggested opening a discussion for the bigger tracing picture so I'm making it.

Quick status first, since it changed since that issue, mode-1 (EVENT_PC) decode is built and working on my end. It's a pure-Python addition to python/utils/trace/utils.py, I made a decode_event_pc_stream() that handles the full mode-1 opcode set (EventPC plus the shared Start/Sync/DC/Repeat framing). I validated it byte-for-byte against a real hardware capture using an independent reference decoder and got 169 commands matching exactly. So the decode half is solid, and what's left is wiring it into the existing pipeline in the way that fits the project.

That's basically what I wanted to check before I open a PR. I can see three places where mode-1 has to touch your existing conventions, but they all have forks I don't want to guess on.

1. Mode dispatch. Does a mode branch belong inside convert_to_commands / convert_commands_to_json, or should there be separate mode-dispatched helpers that share the framing? I prefer latter since it keeps mode-0 untouched.

2. Where per-tile mode gets read. Mode is a Trace_Control0 field, per-tile. Is parse_mlir_trace_events the right place for pulling that out or does it belong elsewhere upstream?

3. Output shape. Mode-1 is PC-anchored (a point per event) rather than the level-span shape mode-0 emits. I want the JSON to line up with whatever conventions event_ir.py already assumes, so if there's an existing shape I'm happy to match that.

Longer term, once mode-1 lands, mode-2 (INST_EXEC) is the natural next step and fits the same framing. I've got that decode working too, but I had to derive it the hard way and I really don't want to step on anybody's toes. If there's any proper documentation on the INST_EXEC encoding, a clean implementation would be a lot easier than that.

No rush on any of this, I know tracing isn't top of the priority list right now. Mostly I just want to build mode-1 in the shape that's least annoying for you to review and maintain, I'm allergic to gaps in a platform like mlir-aie and eager to help close them. Happy to sketch a PR against whichever answers you give.

[hunhoffe]

Thanks for opening this @FIM43-Redeye! Nice work so far!

Tracing is an area we really welcome contributions in, as while the trace tooling is functional (mostly), we're aware the overall trace architecture/organization/structure could use more work. None of us have particularly strong opinions on what shape we want the rework to take, so this task a bit open-ended; do what makes sense, seems useful, follows best practices, etc.

First, one note: I could be wrong, but I believe event_ir.py has a somewhat legacy structure; the pieces of the tracing infrastructure that have seen recent rework are more likely the convert_to_commands (utils.py), plus convert_commands_to_json and
parse_mlir_trace_events (both parse.py).

1. Mode dispatch — your preference (separate helpers sharing the framing, mode-0 untouched) makes sense to me.
2. Where mode gets read — parse_mlir_trace_events (parse.py:376) is right; it already switches on config addresses. I don't think anything reads the mode field today (Trace_Control0 [1:0], 0x340D0 core / 0x140D0 mem), so the parser implicitly assumes mode-0, and setup.py:498 hard-codes trace_mode (TraceMode.EventTime). Making mode actually configurable is part of this. Threading a per-(type, loc) mode out alongside pid_events and dispatching in parse_trace would make sense to me.
3. Output shape — there's no established PC-anchored convention, so do what makes sense. Some claude-generated brainstorming:

Mode-0 emits Chrome Trace B/E level spans; for PC-anchored points, Chrome's instant event (ph: "i") is the idiomatic fit and renders in the same viewers. Lean that way unless your capture suggests otherwise — keep the data faithful, we can adjust conventions later.

On mode-2 / INST_EXEC: no clean encoding docs to hand you, sorry — sounds like you've already derived more than we have written down. Please feel free to contribute your knowledge both to the code base, but also to the documentation in the programming guide and in programming examples.

Hope this helps! Feel free to continue discussion here.

[FIM43-Redeye]

Mode-1 just landed in #3504—thanks again for the warm welcome, guidance, and reviews!

I’m looking at the mode-2 / Execution follow-up now. The semantic findings and hardware evidence are tracked in #3047. For the first PR, I propose adding a low-level decode_execution_stream(byte_stream) beside decode_event_pc_stream() in python/utils/trace/utils.py.

It would return raw command dictionaries for Start/Stop, E/N atoms, New_PC, LC, Repeat0/1, and Sync, while consuming filler frames. parse_trace() would continue to reject mode 2 for now, since reconstructing execution requires the core ELF. That is a substantially larger, ELF-aware API change, and I would want to discuss its shape separately before implementing it.

Does that fit how you’d want mode-2 exposed?

One provenance question before I prepare code: the decoder implementation is original, but its existing frame taxonomy was informed by read-only disassembly of the proprietary aietools decoder, then validated against controlled hardware captures. Before upstreaming, I plan to produce a hardware-only derivation record covering each field and include the validating fixtures. Would that evidence trail be acceptable for this project, or would you prefer a strict clean-room reimplementation by someone who has not examined the aietools decoder?