Scoping a Chess to Peano test migration

[atassis]

Follow-up to #3405, which @hunhoffe asked me to open. The direction reads as settled on that thread:
Peano is the future, and the thing actually anchoring Chess is aiesim (the ISS), not the compiler. So
this is a scoping of what the migration takes, where that anchor really sits, and an offer to help with
both halves.

The Chess dependency in the test suite is really two different things, and separating them is the whole
point:

1. the compiler (xchesscc), and
2. the cycle-accurate simulator (xca_udm_dbg / aiesimulator).

The compiler half is tractable engineering on the open backend. The simulator is the one piece that is
genuinely Chess-only. Numbers grepped from upstream/main (correct me if my greps are loose):

• 146 test files RUN some form of xchesscc; 68 already carry --no-xchesscc (Peano).
• 61 verify inside the Chess ISS (xca_udm_dbg / aiesimulator); 20 of those are gated
  REQUIRES: aiesimulator, valid_xchess_license. 74 files carry REQUIRES: valid_xchess_license overall.
• The compile-side gap is small and enumerable: 4 PEANO-NOT tests, 1 chess_storage use in test/
  (6 more live in programming_examples), and no UNSUPPORTED: peano markers (though Peano-incompat is
  really tracked through the 74 license markers, not that one, so 0 there is not evidence of readiness).

The compiler half

The docs already say you can skip Vitis and target AIE2/AIE2P with Peano, and the ml/norm example runs on
Peano today. For a Chess-compile test that already has a device run, migration is a flag swap plus running
on the NPU; for the compile/IR unit tests it is a flag swap with no device at all. The honest caveats: real
gaps exist right now (the repo's own block_datatypes README notes Peano's AIE2P backend still crashes on
the bfp16 matmul kernels), and Chess-vs-Peano codegen quality is its own open question. Each gap is a filed
llvm-aie fix, which is the backend area I am already working in.

The simulator half, and it is the real work

About 61 tests do functional numeric verification inside the Chess ISS, hardware-free. I piloted the aievec
bf16_* family (test/unit_tests/aievec_tests/aie2/): the testbench.cc there is a host program (it uses
fopen/fprintf to read and write data files) that runs in the simulator. So flipping these is not a
RUN-line edit, it is a per-test execution-model rewrite that moves the numeric check from the ISS to the
device. That, not the compiler, is where the cost sits.

The tradeoff to name up front, because it is exactly #3405's concern

Those ~61 ISS tests run today with no NPU. Moving them to device verification shifts their load onto the
same scarce device runners #3405 is about. So the pragmatic near-term is to keep Chess for the ISS and
migrate the compiler side, which removes the Vitis requirement from most tests and the Chess-specific
serialization without adding device load.

The keystone, then, is aiesim itself. It is the one piece with no open replacement today, it is what ties
the remaining tests to Vitis, and porting it off that dependency is what would let those tests stay
hardware-free instead of moving onto the runners. I would be glad to help port or reimplement that
hardware-free verification path, and I would rather do it with a working slice than argue it in the
abstract. Functional numeric checks are the tractable part; a native cost model for cycle-level timing is a
separate and harder question I would not want to hand-wave. If that is a direction worth opening, I am happy
to take a first cut and report what the open path can actually reach.

Suggested shape, gradual and reversible

1. Pilot one device-path family end to end, proving the compiler swap and surfacing the first real gap count.
2. Move the ISS families to device verification, one family at a time, each gated green, with the added
   runner load tracked so it does not surprise anyone.
3. Fix the codegen tail (the bfp16 matmul crash, chess_storage, any compile failures) as filed llvm-aie
   issues.
4. Flip the aiecc default to Peano once a family is green, and keep Chess as a supported opt-in (and the ISS
   as-is), so the Vitis dependency becomes optional rather than removed.

Two questions, to size this against constraints I cannot see:

1. Is Peano-default plus Chess-opt-in, with the ISS families moved to device, the direction you want, given
   the device-runner cost that shift carries? Or is the hardware-free ISS coverage something to keep exactly
   as it is for now?
2. If yes, want me to start with one device-path family, proving the swap and the first gap count, before
   anyone commits to the sim-test conversion?

[jgmelber]

Thanks for this writeup! The more work we can shift away from Chess will greatly reduce runner load, we can execute more in parallel as Chess is the real resource hog.

For the questions:

1. Peano is the default, we would like to shift away from Chess entirely. AIE1 doesn't have Peano support and new devices take some time to come online in Peano. That is the gap Chess fills.

2. Please do! Your choice on AIE2 or AIE2P.

[hunhoffe]

For the sake of patching together previous work:

• Related issue: Use Peano to build mlir-aie from source #2247
• Related PR draft: [WIP] Enable build mlir-aie and run tests without Vitis #2288

I don't think either of them are being worked on at the moment, but they may provide some useful information.

[thomthehound]

Technically speaking, Windows already proves that mlir-aie can be built from source using Peano. In fact, if I remember correctly, it might possible to cross-compile from Windows right now. I think I at least partially provisioned for that. If not, it's going on the task list.

[hunhoffe]

I haven't done the due diligence to see if they can be fully closed or not, but what you said makes sense -- they are pro0bably outdated.

[thomthehound]

It's my fault, too. I did have those in my notes to request closure once I finished the lit proof. But things fell behind.

[thomthehound]

@atassis I'd just like to express to you my sincere gratitude that you are taking this up. It does not look like an easy task, but it is one that I think will bear great fruit.

[atassis]

Well, that's just a small start. I have to spend much time on my primary job now + few other things happening in life, so it might take a while for me to introduce more interesting work.
It definitely helps that I can talk to someone and get the feedback, even your xdna-driver is opensourced! This definitely helps; have found few bugs there, too. Don't have to work with a blackbox and pray it works well.