JSON-based fuzz tests

[raddad772]

Hi! I maintain some JSON-based fuzz tests for various processors ( https://github.com/raddad772/jsmoo/tree/main/misc/tests/GeneratedTests ) including the Z80, based on TomHarte's tests ( https://github.com/TomHarte/ProcessorTests ) which include 6502.

My test generation is all done in JavaScript, and although it's pretty good, it's not perfect. I had the idea that maybe it would make sense to combine our powers and create the closest thing possible to hardware-based fuzz tests, using the Visual simulators!

Is it possible to use the Z80 and 6502 source kind of like processor cores? In order to generate tests like this, I'd need..

1. The ability to cycle the core 1 cycle at a time (or 2 half-cycles)
2. The ability to read and write all registers and relevant internal state
3. The ability to intercept and replace any memory and I/O reads and writes

In essence, a test is created by choosing an opcode and writing it to a random part of memory (which is treated as flat 64K of RAM). The processor core is setup with random starting conditions except pointing to the next opcode. As memory or IO reads come in, they are serviced with random data, which is recorded, and any writes or relevant pin activity is also recorded each cycle. So you get an initial state for the processor and RAM, the expected bus activity for the length of the instruction, and a final state for the processor and RAM.

These tests are a huge help to emulator authors, both before you have enough of a system going to run a ROM-based test, and also for the final word if you're trying to verify your CPU core's accuracy.

Would you like to collaborate?

[floooh]

Hi, I did something similar using the same simulation code used here in the v6502r project in my 'regular' chips emulator project. The common code is the perfect6502 project (https://github.com/mist64/perfect6502), which might also make the most sense in your case.

Here's my 6502 test for instance which checks the state of my 6502 emulator against the perfect6502 simulation state:

https://github.com/floooh/chips-test/blob/master/tests/m6502-perfect.c

In general, perfect6502 has all the requirements you need, and it can be used with any CPU netlist from the visual6502.org project (in case of Z80 you'd need to integrate this fix into the perfect6502 netsim code:

514a58b

...also see this ticket for details (mainly the comments towards the end, the ticket was written under a wrong assumption).

#2

As for collaborating, I won't have the time :) Typically I only reserve a couple of weeks each year working on retro stuff, and it's entirely random what I'll be working on in that time.