Keyboard input delay/locking/queuing in Gods (Bitmap Brothers)

[FatBarry]

Are you using the latest Dosbox-Staging Version?

• I have checked releases and am using the latest release.

Different version than latest?

No response

What Operating System are you using?

Windows 11

If Other OS, please describe

No response

Relevant hardware info

Gigabyte AORUS 17 XE4 (Core™ i7-12700H, RTX 3070 Ti)

Have you checked that no other similar issue already exists?

• I have searched and not found similar issues.

A clear and concise description of what the bug is.

The issue has only been identified when playing Gods (Bitmap Brothers). No issues identified with other games played so far.

As mentioned in Discord: https://discord.com/channels/514567252864008206/626653390830698516/1242998746153877534

Details:

• Keyboard inputs appear to be delayed and/or queued up (multiple actions despite pressing key once) and sometimes locked (had one game where "left" was locked).
• The issue doesn't always occur immediately - sometimes it can take 30 seconds or so before occurring.
• The issue didn't appear to be present in DOSBox Staging 0.80.1, DOSBox 0.74-3 or DOXBox-X 2024.03.01
• Issue occurs in both fullscreen and windows
• Changing output, glshader does not seem to affect issue.

Steps to reproduce the behaviour.

Explain how to reproduce

1. Using default DOSBox Staging 0.81.1 config

Download URL of affected game or software

No response

Your configuration

https://gist.github.com/FatBarry/5eb96740971a3e075de1cb86cb66ca3c

Provide a Log

No response

Code of Conduct & Contributing Guidelines

• Yes, I agree.

[codengine]

Can confirm that this happens since 0.81.1 (probably 0.81.0 as well) and in latest dev build, it does not occur in 0.80.1.
For me the easiest way to reproduce the issue is just to walk to the right in the first room and alternating between walking left and right. I can also confirm that if you hold an arrow key pressed there is a delay until the next action is invoked.

[codengine]

By the way if one gets stuck in the intro, just use LOADFIX. It doesn't happen all the time for some strange reason.

[johnnovak]

Paging @FeralChild64 for the keyboard regression. It's most likely caused by your grand keyboard refactor 😄

Also, this is 0.81.2 patch release-worthy material.

[FeralChild64]

If I see correctly, this game used to work correctly in 0.81.0 - but on the current main there is a problem like described in the ticket.

[edit] And looks like reverting 0382ab6 makes the problem disappear.

[interloper98]

Maybe your improvements @FeralChild64 are OK, because Gods apparently doesn't handle fast typematic keystrokes on real hardware:

https://www.vogons.org/viewtopic.php?t=76407

Could it be that 0.80's keyboard code was not as accurate as 0.81, but some buggy games had "accidental benefits"?

Bitmap Brothers released "godsfix" patch, mentioned here: https://www.dosgames.com/forum/viewtopic.php?t=3524

But unfortunately the bitmapbrothers website is gone and wasn't archived by the Wayback machine 😞

I guess a typematic control tools is the best bet?

[FatBarry]

If I see correctly, this game used to work correctly in 0.81.0 - but on the current main there is a problem like described in the ticket.

[edit] And looks like reverting 0382ab6 makes the problem disappear.

Sorry, just to clarify - worked fine in 0.80.1, but not in 0.81.1.

I haven't tested 0.81.0, but looks like others have and had the same issue?

[FeralChild64]

@interloper98 We'll see. The game reads the keyboard using INT 16. It constantly calls it with AH=0x01, and - if the key is there - calls it with AH=0x00 to get the scancode.

[interloper98]

I haven't tested 0.81.0, but looks like others have and had the same issue?

@FatBarry oops, yes, I also meant 0.80.1 when I mentioned "0.80".

Checking the release notes for 0.80.1, there are no keyboard changes between 0.80.0 and 0.80.1, so I'm guessing they are both OK in Gods.

https://www.dosbox-staging.org/releases/release-notes/0.80.1/

[Grounded0]

Maybe the game was patched on The Bitmap Brothers Compilation (1995):

https://www.mobygames.com/game/16762/the-bitmap-brothers-compilation/

https://archive.org/details/bitmap-brothers-compilation

If i can't find patches for a game then i usually get the compilation CD version because they have the latest patches applied.

[codengine]

Latest version that I was able to find:
https://www.mediafire.com/file/5r2sguqz2ewd03e/Gods_1.01.7z/file

[MasterO2]

If i can't find patches for a game then i usually get the compilation CD version because they have the latest patches applied.

That's a smart strategy.

[Grounded0]

Download page for GODSFIX:

https://web.archive.org/web/20050325195849fw_/http://www.bitmap-brothers.co.uk/downloads/patches/gods/body.htm

[MasterO2]

https://web.archive.org/web/20050325195849/http://www.bitmap-brothers.co.uk/files/gods/godsfix.zip

godsfix.zip

[interloper98]

Woah nice finds! Thanks :-) Tried both versions (Compilation and 1.01) without and then with the GODSFIX keyboard patch applied (it's a DOS self-extractor that overwrites the GODS.INI file), but unfortunately I still get the same results (Windows 11, AMD CPU).

gods_jumpy_keyboard.mp4

[codengine]

@interloper98 I had the same issue as you show in your video in either version of DosBox. The bug that is described here happens ingame.

[johnnovak]

But the patch seems to be for a different issue, so it isn't really surprising that it doesn't fix our regression 😄

If you run the game and it appears to get stuck on Start game, Enter Password or Exit to DOS Screens, you are experiencing key configuration problems, download this patch and the problem will be fixed.

[Grounded0]

Just exhausting out options that there is a software fix available which is good.

[johnnovak]

Some extra notes @FeralChild64:

• The main menu is almost impossible to use before starting the game—the original ticket only mentioned the actual gameplay.
• I appreciate you tried to implement a more "correct" behaviour, but if you based your work solely on specs, that might not be how keyboard controllers in actual reality work. Would it be possible to selectively revert to the previous keyboard code to get our old behaviour back? I remember you had to do a couple of hacks to fix a few games (e.g. at least Ultima VIII, In Extremis, and Tyrian 2000, from memory, but there might have been more), which kinda means to me that old battle-tested DOSBox keyboard handling code was in fact more correct
• I appreciate you introducing this to fix some niche edge cases in Windows 3.x and stuff, but if we lose compatibility with common DOS games, that's a hard sell 😅 I went back to your original PR, and it seems to me the only semi-important user-facing fix introduced by the reworked 8042 emulation is the keyboard getting sporadically disabled when exiting Windows for Workgroups 3.11 at very high cycles settings? I mean, don't set the cycles too high and that's it 😄 For Win 3.x, the 25k to 60k cycles range is realistic and period accurate, you don't need to go higher than that. Honestly, compatibility with specific CuteMouse versions is rather unimportant... So maybe consider backing out the changes if they don't really fix truly important show-stopper issues but introduce problems with games? Not being able to play Gods is a lot more important than that very edge case-y Win3.11 issue...
• Another idea, and you might have already done that: have you looked into the PCem and 86box keyboard handling code?

[FeralChild64]

that might not be how keyboard controllers in actual reality work

I already know this does not work like a real controller - and I can't easily make it work like a real controller, or our BIOS won't be able to talk to it. At least one reason is within the callback.cpp:

	case CB_IRQ1:	// keyboard int9
		phys_writeb(physAddress+0x00,(uint8_t)0x50);			// push ax
		phys_writew(physAddress+0x01,(uint16_t)0x60e4);		// in al, 0x60
		phys_writew(physAddress+0x03,(uint16_t)0x4fb4);		// mov ah, 0x4f
		phys_writeb(physAddress+0x05,(uint8_t)0xf9);			// stc
		phys_writew(physAddress+0x06,(uint16_t)0x15cd);		// int 15

From what I could observe in 86Box, the real BIOS does something more complicated here, it runs with keyboard controller set up in a different way, where the keyboard entry is only being enabled for the duration needed to read the keyboard input. I can probably change this, with some x86 manual opened in another window, but - frankly - I'm afraid to change this code, I have no idea what could break and in which way, I have almost no experience with x86 assembler, and I have no idea what dirty tricks games might use in connection with this code.

Would it be possible to selectively revert to the previous keyboard code to get our old behaviour back?

But which part?

So maybe consider backing out the changes if they don't really fix truly important show-stopper issues but introduce problems with games?

Well, for me the Windows 3.11 problem was VERY visible. And finding games which did break took quite some time... If I revert to the old implementation, the register-level PS/2 mouse implementation will have to be gone. Not to mention I am rather short of time recently (see how slowly it takes me to rework the host OS locale synchronization) and the reversal would probably be by no means trivial.

---

My problem is, that I have no idea yet what precisely confuses the game. It uses two legacy XT BIOS calls to check/read the keyboard status, so far I haven't found any difference in the behaviour of the BIOS. It uses the Intel 8255 chip to reset the keyboard - this wasn't emulated and still isn't. I can make the game working correctly again by removing a single PIC_DeActivateIRQ(get_irq_keyboard()); call, but:

• this would have effectively removed the Tyrian fix
• what does the IRQ1 actually do that the game needs? I have already ruled out the BIOS_KEYBOARD_FLAGSn and BIOS_KEYBOARD_TOKEN - they are the same during game play, all the time. Maybe the game installs it's own hook for IRQ1, on top of the BIOS - I haven't checked it yet.

[FeralChild64]

It looks like something (besides our IRQ1 assembler code) is also reading the port 0x60 - I just don't know yet if it is some DOSBox code, or a game code.

[johnnovak]

It looks like something (besides our IRQ1 assembler code) is also reading the port 0x60 - I just don't know yet if it is some DOSBox code, or a game code.

Hopefully you're on to something! 😄

Btw, you already know x86 asm, you just don't realise 😏 It's one of the simplest assembly variants, and about 100x simpler than C++. It would take you reading a "Learn X in 15 minutes" article about it to get to intermediate level 😆 I learned it years before C, and even C was super complex in comparison.

So poke x86 code with confidence 😎

[FeralChild64]

Btw, you already know x86 asm, you just don't realise

Not really. I'm not really familiar with the CPU flags, addressing modes, I have no idea how exceptions work, several mnemonics are a mystery to me. And all these CPU modes, segment+address, ugh.

[johnnovak]

Btw, you already know x86 asm, you just don't realise

Not really. I'm not really familiar with the CPU flags, addressing modes, I have no idea how exceptions work, several mnemonics are a mystery to me. And all these CPU modes, segment+address, ugh.

@FeralChild64
Okay, it might seem overwhelming if you're reading the Intel manuals or modern x86 asm tutorials and books. x86_64 has grown into a disgusting Godzilla monstrosity; don't even attempt to go there...

DOS-era x86 asm, on the other hand, is simple and joyful. And primitive compared to 68k with its 1325 address modes 😅, so there's not too much to learn.

Segment:address is dead simple, come on now 😄 You can write C++ code, that's a lot more complex task.

Ignore protected mode completely, just focus on 8086 stuff. You then only need to familiarise yourself with some 386 instructions and the extended 32-bit registers, but that's it. Stay away from protected mode, privilige modes, paging, exceptions as if it were the plague or some STD 🤣 -- you're not writing an OS, neither a DOS extender. Very few people understood that advanced stuff back in the day. I happily coded in asm until the end of the 90s without understanding anything of that, and so did most coders.

Here are some links I researched this morning because I also need to brush up my asm skills a little. But strictly DOS era; the modern 64-bit stuff is horrific...

1. One modern tutorial that isn't half bad. Just ignore the 64-bit and protected mode parts:
   https://www.codeproject.com/articles/45788/the-real-protected-long-mode-assembly-tutorial-for

2. Read the first two chapters of The 8086/8088 Primer. Free on the author's website:
   https://stevemorse.org/8086/index.html

3. Read this very good asm tutorial:
   http://textfiles.com/programming/asmtutor.pro

4. Optionally read chapter 3 of the primer.

5. Refer to the short instruction descriptions in Help-PC as you work on some asm code. I used the original TSR version back in the day, it's indispensable. Contains opcodes up to the 486. The timings were super handy too when optimising stuff.
   https://docs.huihoo.com/help-pc/

Do this and you'll be a classic x86 asm guru in no time! 😎

Further oldskool x86 asm materials:

• https://www.oocities.org/siliconvalley/2151/pcgpe.html
• https://mattst88.com/AssemblyProgrammingJournal/issue/1/

[gobalius]

@interloper98 I had the same issue as you show in your video in either version of DosBox. The bug that is described here happens ingame.

How did you manage to fix this issue? I have this issue exactly, the one shown in interloper98's video. Almost impossible to select "ENTER PASSWORD". Everything else works as it should. Oh, and I tried both 0.80.1 and 0.81.1, and also other dosbox variants, but it's all the same everywhere (some other dosbox variants even have screen flickering in the menu)...