[Tracking issue] Test infrastructure for Stage 2

[aborg-dev]

This is a tracking issue for all tasks related to test infrastructure that we want to complete in Stage 2.

For Stage 2 we aim to support the majority of WASM instructions, not limited to basic i32 and i64 operations.

To make sure our implementation is correct, we will expand our test suite to more tests from WASM testsuite.

As the test suite grows, we must invest in faster and more convenient tooling to run the tests.

Given the scope of the work, it might be worth writing a short design document describing how the testing would work end-to-end.

Tasks

Beta Give feedback

1. Support different types of asserts in tests. #145
   +0
2. Check that test_summary.csv is up-to-date in CI #133
   +0
3. Migrate to Wasmtime filetests
4. Add tests covering memory.wast
5. Add tests covering data.wast
6. Add tests covering table.wast
7. Speed up CI by disabling unused Rust tests
8. Speed up CI by caching NPM install
9. Add globals spectest #165
   +0
   

[MCJOHN974]

Some updates about this work:

1. Adding spectests is no still actual due to chosen implementation strategy.
2. Disabling unnecessary tests. In discussion, no final decision here. Related issue
3. Migrate to cranelift filetests -- in active work, maybe it make sense to split it into several issues to track the progress, task is relatively big.
4. Support different types of asserts in tests -- formally not finished, but introducing zkasm helper for assert made a good progress on it, now it is easy to simply add new types of assert here. Also, we don't really need all this asserts asap with migrating to filetests.
5. Checking test summary.csv in CI -- blocked by filetests migrating
6. Speed up by caching npm install -- still planned and not prioritized. Is it a good topic for zkasm programming session?

[MCJOHN974]

Also, here is more actual roadmap of testing infra development:

• Make decisions on filetest migrating (Done)
• Implement skeleton of zkasm filetest (Done)
• Fill skeleton with some mock parts, implement an easy parts and check that it really can parse and run .clif file with test. (Done)
• Compile .clif run command with proper register allocation (In progress)
• Adopt generate_preamble and generate_postamble functions if needed (TODO)
• Process assert helper output and build results.csv from it (TODO)
• Add filetest infra to CI (TODO)
  ===========================
• Speed up CI by caching, disabling some tests, etc.
• Research which enrichments to such test infra we need in nearest future.