z80: interruptible execution

[holub]

Credits:

• undocumented flags - Manuel Sainz de Baranda y Goñi, Peter Helcmanovsky, Patrik Rak
• tables removal - smf, happppp, Lord Nightmare

z80 interruptible implementation is ready for review
Main execution don't overflow scheduler frame more than 3 icounts (atomic for IO read/write)
take_interrupt() hasn't been converted yet (possibly TODO)

Drivers which must be updated before all TIME_GUARD sections completle removed from the code:

• Amstrad
• MSX
• System1

[holub]

More confident now... Let's try to make it happen!

[holub]

drives identified as broken with PR:

• sigmab98.cpp - all machines

[holub]

All not working drivers identified during testing fixed!

[simzy39]

Can you mark this as Ready for Review?

[holub]

Can you mark this as Ready for Review?

I think it is. Yet actuated with current master.

[ghost]

There seem to be 2 lines of thought with this, yet surprisingly nobody leaving comments here.

The first is that the Z80 code is already ugly, relies too much on macros, and trying to fit these changes in a core never designed for it is a waste of time and we should just wait for a complete new core.

The other is that this is the best way to get these changes, as it built on top of a mostly trusted Z80 core, so if things break in the process it'll be easier to identify and fix them, whereas a brand new core would almost certainly come with it's own set of problems at first that might make it difficult to identify the source of issues.

I'm in the latter camp, I think this is worth doing, but maybe the lack of other comments / movement suggests that is a minority opinion.

[holub]

I agree lack of comments on this PR is discouraging.
IMHO we can wait for new z80 core for another 15 years or take advantage from these changes immidiatly. It's already provides big step forward and I think using it I can reimplement Spectrum memory contention using WAIT state, without playing with icount.
Macroses are ugly but without them the changes look even worse with anonymous functions. And frankly a lot of original macroses removed here.

I open to any suggestions if anybody is interested.

[smf-]

I don't know what the performance difference is, but I still think with the macros that this will be easier to deal with than a python code generator.

…

On 30/08/2022 20:21, David Haywood wrote: I'm in the latter camp, I think this is worth doing, but maybe the lack of other comments / movement suggests that is a minority opinion. Message ID: ***@***.***>

[holub]

We should expect performance drop because old code was highly optimized with inlining which is not possible in inerruptible version. Based on my naive "max frame skip" performance test, the drop is ~40% from original.
That is still acceptable for z80 as it's not expected to run with clock faster than 20MHz.

[ghost]

ultimately this is going to be the decision of @cuavas

will you allow this route to be taken? aside from a performance decrease (which will likely happen even if a new cycle-accurate core is written from scratch) this doesn't inherently seem worse than what we have on a technical level.

personally I'd like it if at the point we get python generated cores, that it be possible to generate 2 different cores, one 'performance' core, where we're confident that cycle accuracy never comes into play, and another where it does, in order to reduce any performance loss for the less demanding cases, but if such a core is written we can deal with it at the time.

as it stands there are a number of cases where improving the Z80 is going to be important and could lead to improvements in the emulation of other components.

[ghost]

@cuavas since no consensus can be reached on this, how about some kind of compromise, like including it as a secondary core, /z80_interruptible or similar?

that way, it can be used where there seem to be benefits, with no risk to anything else, and can prove its worth in those situations?

[rb6502]

I think it's fine to actually just be the Z80 core. I'm not really concerned about perf on any Z80 drivers. Would be nice to get a second opinion from @cuavas or @galibert first though.

[holub]

Custom tables only left in the system1. Need inspiration or help...

[simzy39]

Are you a member of the forums? Can post threads there or chat in the chat box. https://forums.bannister.org/
Great work tho, hopefully @cuavas can give feedback

[holub]

added as a comment - sharing for discussion here:

system1 hardware change clock for m1. There are two possibilities to get rid of custom tables here (suprloco requires the same).
1. Keep overclocked z80 and use multipliter + adjust icount respectively in m1 (opcode_read)
	pros: more precise timings
	cons: z80 requires extra multiplier state which no other drivers use
2. Use proper z80 clock and only adjust icount in certain m1 reads
	pros: z80 can drop multiplier support
	cons: for couple commands timings going to be behind actual

[holub]

I can declare a victory now!

[holub]

'Q flag' for ccf/scf fixed:

[holub]

If there are concerns regarding this PR and we still don't see it happen, I can move tables removal and flags update to master branch.

[holub]

all tests in Patrik Rak's (raxoft) z80test - passed 100%
+ rest interrupted block instructions uncovered by them:

[holub]

I'm currently reimplementing the 68000 for. among other things, interruptibility. Once it fully works I'm going to add all that's needed for wait states & friends (which shouldn't take that long) and test it throughly on the ST and the Lisa. What about we wait on the z80 until then (eg ~ a month) to see how we can make it fit in what's needed for the wait states?

@galibert your turn

[simzy39]

What`s the performance drop now?

[holub]

What`s the performance drop now?

If I did everything right... ~30%:

» ./mame-master spectrum -bench 300
Average speed: 2768.18% (299 seconds)
------------------------------------------------------------------------------------------------
» ./mame-int spectrum -bench 300
Average speed: 1878.82% (299 seconds)

[rb6502]

This cannot be compiled on a Mac with 16 GB of RAM and no other applications open. That's a serious problem.

[rb6502]

John IV reports "pacman -bench 90 from 18110 to 5749". That's 30% of the former speed, which is also a serious problem.

[john-iv]

Further: Mag_xain 488 to 234, 52% hit Tnk3 4056 to 1524, 62% hit

[holub]

Let's revert it then. I don't have enough technical knowledge to fix these performance issues quickly.

[holub]

@rb6502 @john-iv just to be clear... you were talking about compilation with DEBUG=0 ?

[john-iv]

Yep.
make -Werror REGENIE=1 -j21 STRIP_SYMBOLS=1 ARCHOPTS=-fuse-ld=lld
The numbers were on my i7-12700k at 5.2Ghz