cputester results

[dirkwhoffmann]

Current status (will be updated permanently):

ABCD.B 👍
ADD.B 👍
ADD.L 👍
ADD.W 👍
ADDA.L 👍
ADDA.W 👍
ADDX.B 👍
ADDX.L 👍
ADDX.W 👍
AND.B 👍
AND.L 👍
AND.W 👍
ANDSR.B 👍
ANDSR.W 👍 (zipped, 8 MB Fast RAM)
ASL.B 👍
ASL.L 👍
ASL.W 👍
ASLW.W 👍
ASR.B 👍
ASR.L 👍
ASR.W 👍
ASRW.W 👍

BCHG.B 👍
BCHG.L 👍
BCLR.B 👍
BCLR.L 👍
BSET.B 👍
BSET.L 👍
BSR.B 👍
BSR.W 👍
Bcc.B 👍 (zipped, 8 MB Fast RAM)
Bcc.W 👍 (zipped, 8 MB Fast RAM)

CHK.W 👍
CLR.B 👍
CLR.L 👍
CLR.W 👍
CMP.B 👍
CMP.L 👍
CMP.W 👍
CMPA.L 👍
CMPA.W 👍
CMPM.B 👍
CMPM.L 👍
CMPM.W 👍

DBcc.W 👍
DIVS.W 👍
DIVU.W 👍

EOR.B 👍
EOR.L 👍
EOR.W 👍
EORSR.B 👍 OK (zipped, 8 MB Fast RAM)
EORSR.W 👍
EXG.L 👍
EXT.L 👍
EXT.W 👍

ILLEGAL 👍
ILLEGAL.B 👍

JMP 👍
JSR 👍

LEA.L 👍
LINK.W 👍
LSL.B 👍
LSL.L 👍
LSL.W 👍
LSLW.W 👍
LSR.B 👍
LSR.L 👍
LSR.W 👍
LSRW.W 👍

MOVEx.B 👍 (zipped, 8 MB Fast RAM)
MOVEx.L 👍 (zipped, 8 MB Fast RAM)
MOVEx.W 👍 (zipped, 8 MB Fast RAM)

NBCD.B 👍
NEG.B 👍
NEG.L 👍
NEG.W 👍
NEGX.B 👍
NEGX.L 👍
NEGX.W 👍
NOP 👍
NOT.B 👍
NOT.L 👍
NOT.W 👍

OR.B 👍
OR.L 👍
OR.W 👍
ORSR.B 👍
ORSR.W 👍

PEA.L 👍

RESET 👍
ROL.B 👍
ROL.L 👍
ROL.W 👍
ROLW.W 👍
ROR.B 👍
ROR.L 👍
ROR.W 👍
RORW.W 👍
ROXL.B 👍
ROXL.L 👍
ROXL.W 👍
ROXLW.W 👍
ROXR.B 👍
ROXR.L 👍
ROXR.W 👍
ROXRW.W 👍
RTE 👍
RTR 👍
RTS 👍

SBCD.B 👍
STOP 👍
SUB.B 👍
SUB.L 👍
SUB.W 👍
SUBA.L 👍
SUBA.W 👍
SUBX.B 👍
SUBX.W 👍
SWAP.W 👍
Scc.B 👍

TAS.B 👍
TRAP 👍
TRAPV 👍
TST.B 👍
TST.L 👍
TST.W 👍

UNLK.L 👍

[dirkwhoffmann]

For reference: CHK.W (N flag is wrong):

[mithrendal]

Disk full: you could spread the files on two disks and copy the contents of them to ram: together with the cputester

Then you should be able to start cpu tester from ramdisk

Copy df0: to ram: ALL

[dirkwhoffmann]

😎

[mithrendal]

🤤🤤🤤

[dirkwhoffmann]

Disk full: you could spread the files on two disks

Using the RAM disk is a good idea. As I understood, Toni is also working on gzip support for the .dat files. This should solve the issue for most test cases (the dat files are heavily compressible).

[dirkwhoffmann]

Toni has built in gzip support which solves my problem 👍.

Test case ANDSR fits on a single ADF now and runs just fine in vAmiga:

cputester really pays off. I did expect issues with DIVS and DIVU, but not with LINK 🤭. Apparently, Moira writes a wrong stack frame 🙈:

[tonioni]

LINK A7,#something makes no sense but it is still technically valid instruction. Note that different CPU models handle this differently. ("LINK stacked value is saved before SP is decreased by 4. Only affects pointless LINK A7,#x variant. (All models except 68040)" says my notes)

[dirkwhoffmann]

The issue with the LINK instruction was actually a simple one. In my code, I explicitly subtract 4 when register A7 is used:

template<Instr I, Mode M, Size S> void
Moira::execLink(u16 opcode)
{
    int ax   = _____________xxx(opcode);
    i16 disp = (i16)readI<S>();

    push<Long>(readA(ax) - (ax == 7 ? 4 : 0));

    writeA(ax, reg.sp);
    reg.sp += (i32)disp;

    prefetch<LAST_BUS_CYCLE>();
}

I did this, because I reimplemented what I saw in PiCiJi’s portable68000/Denise emulator. He implemented it that way.

I’ll let him know that this is wrong. A7 needs to be handled just as every other register.

And indeed. Changing line

push<Long>(readA(ax) - (ax == 7 ? 4 : 0));

to

push<Long>(readA(ax));

fixes the issue 😎.

[dirkwhoffmann]

Finally. I've run all tests by now 🥵.

Unexpectedly, Moira only has a very few issues left:

• ANDSR.B 😕 Needs a 1.44MB disk (zip files too large)
• BSR.W 😕 Needs a 1.44MB disk (zip files too large)
• DIVS.W 😫 FAIL
• DIVU.W 😫 FAIL
• EORSR.W 😕 Needs a 1.44MB disk (zip files too large)
• ORSR.W 😕 Needs a 1.44MB disk (zip files too large)
• Scc.B 😕 Needs a 1.44MB disk (zip files too large)

Assuming that the five untested test suits pass, only the DIV command has issues (which should be easy to fix) 😃.

Regarding the five too large test cases, I can split Scc.B into 8 separate ones (one for the 8 possible addressing modes). The others can’t be split, because they use a single addressing mode. I’ll run them inside the RAM disk tomorrow (after reading in the files from different disks). Maybe I should really add support for 1.44MB drives which would solve the issue, too. Hard-drive support is difficult I guess, so I better keep my hands away from this for the moment.

Time for a beer 🍺 …

[tonioni]

I added quick and dirty extra option: askifmissing. if it is enabled and xxxx.dat file is not found, it asks for path and retries. Note that because there is no "this is really the last file" marker (at least not yet), it will also ask for new path even if there is no more files remaining. Empty path = exit.

[mithrendal]

Ah clever, that means when cputester reports about the missing file, dirk can swap disks in the background ... hehe😏

[dirkwhoffmann]

I appended askifmissing=1 to the end of my .ini file, but it didn’t ask me for a path. 🤔

What about adding something like a path variable instead of askifmissing? If a file isn't found relative to the current directory, cputester could try again by searching in path.

Anyway, I was stupid yesterday. I simply had to reduce the number of test runs from 2 to 1 to make the (zipped) test cases fit onto a single ADF 🙄.

I have also seperated Scc into multiple test cases by splitting them w.r.t. the applied addressing mode. By doing this, I noticed that cputester didn’t generate test cases for the PC-relative addressing modes PC16 and PC8r. Is this a bug?

[tonioni]

askifmissing is m68k test program command line parameter ("cputest instruction.x askifmissing"). It has nothing to do with generating tests.

Scc does not support PC16/PC8r variants. (ILLEGAL test includes these)

[dirkwhoffmann]

Meanwhile DIVU and DIVS are passed, too. This means that Moira passes 100% of all tests by now 🥳.

Overall I think this is a shining example of the strength of open source software. With the existence and free availability of Musashi, portable68000 and cputester I was able to implement a working 68000 CPU within weeks. I am amazed myself about this short development time.

Next step: I will further clean up Moira and put it into a public repo on Github. Maybe others can benefit from this as well.

Thanks to everyone who helped me with this!!!

[tonioni]

Nice. Next step is trace :)

btw, I added optional automatic gzip compression support to test generator. (Binary not yet available, need to do some testing first because of cpu emulation related changes)

[mithrendal]

🙋🏻‍♀️Toni, what does "next step is trace" mean ?

Two other aspects for the test of an CPU Emulation come to my mind ...

• first instruction total cycle length and

• second the bus cycle length of the intermediate read/write DMA-bus access of an instruction.

Do you maybe also have an idea or hint about this ? 🤓

I read this in your description text for cputester

Instruction cycle order or timing is ignored. It is not possible without extra hardware.

Currently my understanding is that it is still possible to test the first aspect with software 🤗 as dirk did already here but the second aspect the intermediate bus access is extremely difficult to test 🤢 with software alone, ... see some thoughts here where dirk also thinks it is hardly doable in a fine grained way (search for "intermediate bus access") Maybe only possible with a thing like a logic analyser 🥴?

[tonioni]

Trace as in Trace mode (SR T-bit). Many protections use it ("trace vector decoder") and not all cracks remove it.

Getting accurate cycle length is not possible (almost correct should be possible) in real world. Only way is to use logic analyzer (I used logic analyzer to implement accurate timing over 10 years ago or so). Another possibility is to analyze recently released fx cast core which is based on 68000 microcode. (But then you probably hardwire your emulation to 68000 only. Which can also be valid choice.)

But in my opinion instruction cycle count length can be only used to check that instruction is not totally wrong in emulation, using it to do regression testing. Order of memory accesses and positions of any possible "idle" cycles are very important because it will affect timing greatly if CPU is accessing chip ram and Agnus DMA is also active. Instruction cycle count results without any DMA contention might look nice and match documentation but don't have much common with real world situation.

[dirkwhoffmann]

Trace as in Trace mode (SR T-bit). Many protections use it ("trace vector decoder")

That's good to know. I formerly thought that the trace functionality was only used for hardware debugging. I'll put trace support on my TODO list 🤤.

[dirkwhoffmann]

Nice. Next step is trace :)

vAmiga until yesterday: 😲

vAmiga today: 🥳

I have to admit that I didn't run all tests 🙄. Just a single one: OR.W with feature_sr_mask=0xa000.

[tonioni]

I recommend also testing SR modifying instructions. (ANDSR/ORSR/EORSR,MOVE to SR,STOP,RTE).

btw, cycle count mode is coming soon. I thought it wasn't possible to have accurate enough cycle counting without custom hardware. I was wrong.

[dirkwhoffmann]

I recommend also testing SR modifying instructions.

Thanks, I will 👍.

btw, cycle count mode is coming soon. I thought it wasn't possible to have accurate enough cycle counting without custom hardware. I was wrong.

cputester is already a great tool, but this one is gonna be a killer feature. I'm looking forward to get my hands on the new version 🤤.

[dirkwhoffmann]

Hi Toni,

I just downloaded the latest cputester revision (using the link on the first page of the EAB thread).

When I run it in my Windows VM, it crashes after printing the parsed options:

If needed, I can upload the .ini file.

[tonioni]

Does default unmodified ini also crash? Does any more log files appear if you replace mode=all with mode=or.l ? (or set verbose=2)

[dirkwhoffmann]

Default .ini file works.

It crashes when I comment out test_low_memory_start and test_low_memory_end

; Low address space limits. Real hardware must have RAM in this space. Comment out to disable.
; Start should be zero if Amiga, set to 0x0800 if Atari ST.
; Must be disabled if cycle counting, cycle count tests must only access real Fast RAM.
;test_low_memory_start=0x0000
;test_low_memory_end=0x8000

[tonioni]

I can't duplicate it but I'll debug it more later today. It isn't that important because you need to disable "low memory" anyway if you want correct cycle count results.

EDIT: Oops, I read it the wrong way (works if disabled), now I can duiplicate it :)

[tonioni]

Should be fixed now.

[dirkwhoffmann]

In the latest revision, cputester seems to ignore my modesetting.

My config files has

; mnemonics separated by comma or all/fall.
; all = generate all CPU tests. tst = generate tst.b, tst.w and tst.l. tst.l = generate only tst.l
; fall = generate all FPU tests.
; branch = all branch instructions (branchj = non-stack only, branchs = stack using)
mode=and

but it generates tests for all instructions.

Do I have to enable cycle count checking explicitly or is it enabled by default? I haven't found an option for that.

I've also noticed that the new .ini file comes with various default settings such as

[test=IRQ]
enabled=0
mode=nop,ext,swap
feature_interrupts=1

How can I tell cputester to use one of these settings? 🤔

[tonioni]

Cycle count is always enabled if 68000 (or 68010 but results are useless). m68k test program has option to either ignore included cycle counts (default) or enable it (-cycles)

Ini structure has changed, [cputest] section does not anymore generate anything. Only [test=] generate tests. Enabled=0: don't generate tests (This makes it easy to have multiple "profiles" in same ini). Changed it to enabled=1 and you'll get 68000_IRQ directory containing test data.

"Default" test is the one that m68k tester runs by default. Other test sets/groups/profiles (whatever you want to call them) need cputest < test group name >/< name of instruction or all >.

[dirkwhoffmann]

Ini structure has changed...

OK, got it.

Now, I am experiencing another problem. I created tests for TST.W, but they fails in both UAE and vAmiga. I guess it has something to do with test generation and not with the emulators.

What I did is to shrink the test memory size to

test_memory_size=0x40000   

in the .ini file to make the test case fit on a single ADF. Could it be related to that or do you think it is something else?

[tonioni]

Why not just disable slow ram completely? (chip ram + 8M fast ram only) Or use fastmemfirst in WB disk but I don't think it is worth the trouble..

[dirkwhoffmann]

Why not just disable slow ram completely?

Mission accomplished 🥳. With SlowRam disabled, the tests runs just fine in UAE:

I am curious to see how vAmiga will perform. Obviously, I need to fix VHPOSR first...

[tonioni]

You can also "cheat": implement word size register somewhere in memory space that counts cpu cycles and call cputester with -cyclecnt

(for example -cyclecnt 0xf00000 if cyclecouter is located at 0xf00000), this replaces use of dff006. Address must have normal fast ram access timing.

btw, cputester small update: reported cycle count is now separated in to instruction part and exception part.

[dirkwhoffmann]

Making progress...

Test case TST.W with the old version of vAmiga (v0.55, faulty VPOSR):

Test case TST.W with the latest and greatest code changes 😎:

All I had to do was to fix the implementation of Agnus::peekVHPOSR() according to your remarks above:

uint16_t
Agnus::peekVHPOSR()
{
    // 15 14 13 12 11 10 09 08 07 06 05 04 03 02 01 00
    // V7 V6 V5 V4 V3 V2 V1 V0 H8 H7 H6 H5 H4 H3 H2 H1

    if (pos.h > 1)
        return BEAM(pos.v, pos.h) & 0xFFFF;

   if (pos.v == 0)
       return BEAM(isLongFrame() ? 312 : 313, pos.h) & 0xFFFF;

    return BEAM(pos.v - 1, pos.h) & 0xFFFF;
}

Next step will be to test the other instructions as well...

[dirkwhoffmann]

Excellent tool 😎. It just told me that I've got a timing problem in MOVE <ea>,SR.

Bcc.b and Bcc.w are good though.

[dirkwhoffmann]

Now I'm getting into trouble interpreting the result:

I understand that vAmiga is consuming too many cycles here, but how exactly do I have to interpret the numbers written in parenthesis (46+34)?

Update: vAmiga triggers an unexpected trace exception here.

[tonioni]

Does same test pass without -cycles?

(x + y) means: y = cycles taken by "exit" exception. x = cycles taken by test instruction(s).

It probably is trace related (moving #$ffff to SR), trace not getting disabled when illegal instruction starts (some exceptions clear it). Trace is a special case in cycle counting, whatever exception comes first will stop cycle counting. Most likely illegal instruction was generated (as expected) and then trace started immediately (which is not correct in this situation).

[dirkwhoffmann]

It probably is trace related

Yes, it's trace related. I've generated another test case with trace flag / supervisor flag settings and ignored cycle counting. The test fails, because the wrong exception is executed in vAmiga. Instead of triggering a privilege violation, it triggered a trace exception.

[dirkwhoffmann]

trace not getting disabled when illegal instruction starts

Nailed it. I did clear the trace flag, but I didn't clear the internal action flag which is used to trigger the trace exception handler at the right time.

Thank you so much for all the good advice. That saved me a lot of debugging time 👍.

[dirkwhoffmann]

Hi Toni,

I just computed tests for the TRAP instruction with T flag enabled. In UAE, I get the following error:

Do you have an idea what could go wrong here?

[tonioni]

It is tester bug, it seems some exception + trace combinations don't work correctly yet.

Fix will need to wait because currently I am adding 68010 cycle count support and there are some non-trivial changes needed to support it properly.

[dirkwhoffmann]

Fix will need to wait

No worries. I'll be busy for a while testing the other instructions. BTW, vAmiga displays the exact same error as UAE which I interpret as a good sign. And if cycle checkin is disabled, the test passes (which is definitely a good sign).

[dirkwhoffmann]

There is also an issue with the test of the JMP instruction (with and without cycle counting). I get an error in UAE when the supervisor flag is set:

[dirkwhoffmann]

Oups, I now got an "ahist overflow" 🤭

[mithrendal]

maybe just an overwhelming overflow of stunning, compelling, cool, new and modern testing stuff which is constantly being added by toni to cputester, breaking with everything which the world has seen before 🤤🤤🤤... see definition of ahistorical

[dirkwhoffmann]

It's the MVMLE.L instruction. The overflow also happens if this test is created alone.

[dirkwhoffmann]

Very cool 😎. cputester showed me that my MOVEP implementation is wrong. Moira falsely triggered an address error when an odd address was given.

Same for MOVEM. Moira always triggered an address error when an odd address was specified. But the address error does not trigger if the register mask is 0 (I never thought about the case somebody could (theoretically) use this command with a zero 0 reg mask, because it doesn't make sense).

[mithrendal]

It's the MVMLE.L instruction. The overflow also happens if this test is created alone.

what a strange instruction I never have seen it ... I wonder how many of these seldom seen instructions are actually being used by compilers when they compile to asm. I bet they only use a subset of all possible instructions. Is this the difference between RISC and CISC processors? CISC processors developers tend to invent special seldom used instructions. Maybe they had in mind to substitute a code sequence of the more common instructions in order to save some memory space ??? 🤔

[tonioni]

MOVEM has also previously used too many "history" entries. I'll also check it after 68010 stuff is more complete. You need to create later tests manually without "all".

MOVEM zero mask is sort of edge-case but it actually exists in real world, some early crap C(?)-compilers have created it. I have seen it some early Amiga binaries but I don't think it has ever been used to generate/not generate mysterious address errors.

Actually I also didn't know this behavior until tester reported it :)

MVMEL and MVMLE = MOVEM to registers and from registers. Technically they are different instructions and need different label. (Don't ask why those labels, they already existed when I started doing UAE stuff ages ago)

[dirkwhoffmann]

Interesting. The Motorola ref manual states that only bits 0 - 4 have a meaning in the lowest word of the address error exception stack frame.

Apparently, this is not the case. For two different test cases, the expected value is $4ed2 and $4ef2, respectively:

Both values differ in bit 5, so this bit must have a meaning, too. Some undocumented feature of the M68000? 🤔

[dirkwhoffmann]

This should be the place where UAE computes the "mystery word" located at the lowest stack frame address.

		if (nr == 2 || nr == 3) {
			// 68000 address error
			uae_u16 mode = (sv ? 4 : 0) | (last_instructionaccess_for_exception_3 ? 2 : 1);
			mode |= last_writeaccess_for_exception_3 ? 0 : 16;
			mode |= last_notinstruction_for_exception_3 ? 8 : 0;
			// undocumented bits seem to contain opcode
			mode |= last_op_for_exception_3 & ~31;

So it seems that a part of the opcode shows up in the upper bits (5 - 15). Uhhhh, who could have expected that? 🤭

[tonioni]

Address error frame is full of undocumented behaviors.. I/N bit don't always work as documented, opcode field sometimes contain current, sometimes next opcode and probably more..

But as I said previously, no one cares. Except tester :)

[dirkwhoffmann]

opcode field sometimes contain current, sometimes next opcode

Hmm, yes, I just stumbled across this:

I think I jeopardise my plan to make address error handling 100% correct and move on... since it's not really relevant to Amiga emulation, it really doesn't seem to be worth the effort.

[dirkwhoffmann]

UAE shows a cycle mismatch when testing the JMP instruction in combination with address errors and cycle counting:

I already checked the mem config: 512 KB Chip + 0 KB Slow + 8 MB Fast

[tonioni]

Yeah, it is already known and fixed. JMP and JSR EA calculation is a special case (no normal prefetches) and EA calculation time was done after address error check.

[dirkwhoffmann]

After doing some top-level GUI refinements, I am finally back at cycle counting 🤤.

I am having some weird issue with the latest cputester version though (I guess my configuration is wrong, but I don't know why). My problem is that I cannot get cputester to run if SlowRam is disabled. Running with SlowRam enabled, it runs just fine:

But with SlowRam disabled, it eventually freezes. I don't know why this happens, because (I think) I configured it to place the tests in FastRam, only 🤔. Here are the relevant parts of my config:

test_memory_start=0x860000
test_memory_size=0x40000
opcode_memory_start=0x87ffa0

[test=Default]
enabled=1
test_rounds=2
mode=jmp

I am running UAE 4.3.0 with Chip = 512KB, Slow = 0KB, Fast = 8MB

[dirkwhoffmann]

Found it. It was an issue with my script that didn't update the ADF properly...

[dirkwhoffmann]

Now I am running into issue with the JMP instruction again. With latest cputester, it still fails in UAE when address errors are combined with cycle count checking:

Mem config: 512 KB Chip + 0 KB Slow + 8 MB Fast

Yeah, it is already known and fixed. JMP and JSR EA calculation is a special case (no normal prefetches) and EA calculation time was done after address error check.

Ist the fix part of cputester or part of UAE? In the latter case, it's clear that the test fails, because 4.3.0 came out earlier.

[tonioni]

It is UAE fix.