VGA sequencer emulation fix: when DOS programs write the sequencer index

register (port 0x3C4) most VGA/SVGA cards actually decode fewer bits
than the 8-bit value given. This can be seen through casual poking or
from the Hackipedia VGA register dumps I've posted online, where
Sequencer registers will repeat through indexes 0-255 depending on how
many bits are decoded by the hardware. Currently it is known that ET4000
cards decode only the lowest 3 bits, Trident TVGA cards decode the lowest
4 bits, and S3 cards decode the lowest 6 bits. This behavior means that
"aliases" of common Sequencer registers exist that function exactly the
same as the normal registers.

The reason for this fix, is the 1993 demo "Spiral" by Dataction, which
has a flawed Mode X implementation. It sets up VGA 256-color Mode X as
normal, but instead of writing directly to the Map Mask Register
(Sequencer index 0x02) it writes to an alias of the Map Mask Register
(Sequencer index 0x12) that exists on older VGA hardware (standard VGA,
ET4000, Trident) but not on newer VGA hardware (S3). This is why, when
the demo is run on newer hardware, the Mode X graphics don't render
properly. Adding the sequencer masking behavior allows the demo to
render Mode X properly with machine=svga_et4000 and the graphics no
longer look like a low resolution blurry mess.

src/hardware/vga_seq.cpp

@@ -28,6 +28,52 @@ Bitu read_p3c4(Bitu /*port*/,Bitu /*iolen*/) {
 }
 
 void write_p3c4(Bitu /*port*/,Bitu val,Bitu /*iolen*/) {
+	// FIXME: Standard VGA behavior (based on observation) is to latch
+	//        the sequencer register based only on the lowest 4 bits.
+	//        Thus, the map mask register is accessible not only by index 0x02,
+	//        but also from 0x12, 0x22, etc..
+	//
+	//        This isn't quite universal, as WHATVGA documentation suggests
+	//        there are SVGA chipsets with extended registers in the 0x07-0x17
+	//        and 0x80-0x9F ranges. But register snapshots so far seem to suggest
+	//        most cards just mask off the index to 4 bits and put the extended
+	//        regs elsewhere.
+	//
+	//        The yet to be answered question is: if the card only latches the
+	//        low 4 bits, then what will you see when you read back the sequencer
+	//        index register? Will you see the full 8-bit value, or the low 4 bits
+	//        it actually decoded?
+	//
+	// FIXME: I don't have an EGA to test with, but, what masking behavior here
+	//        do EGA and compatibles do with the index?
+
+	/* Emulating the mask behavior fixes the following problems with games & demoscene entries:
+	 *
+	 * - "Spiral" by Dataction: SPIRAL.EXE appears to use VGA Mode-X 256-color mode, but it relies
+	 *   on sequencer alias register 0x12 for masking bitplanes instead of properly writing
+	 *   Map Mask Register (register 0x02). Older VGA chipsets only decoded 3 or 4 bits of the sequencer
+	 *   index register, so this happened to work anyway since (0x12 & 0x0F) == 0x02, but it also means
+	 *   the demo will not render Mode X properly on newer SVGA chipsets that decode more than 4 bits of
+	 *   the sequencer index register. Adding this masking behavior, and running the demo
+	 *   with machine=svga_et4000 allows the demo to display properly instead of rendering as
+	 *   a blocky low-res mess.
+	 *   [http://www.pouet.net/prod.php?which=12630]
+	 */
+
+	if (machine == MCH_VGA) {
+		if (svgaCard == SVGA_S3Trio)
+			val &= 0x3F;	// observed behavior: only the low 6 bits
+		else if (svgaCard == SVGA_TsengET4K)
+			val &= 0x07;	// observed behavior: only the low 3 bits
+		else if (svgaCard == SVGA_TsengET3K)
+			val &= 0x07;	// FIXME: reasonable guess, since the ET4000 does it too
+		else
+			val &= 0x0F;	// FIXME: reasonable guess
+	}
+	else if (machine == MCH_EGA) {
+		val &= 0x0F; // FIXME: reasonable guess
+	}
+
 	seq(index)=val;
 }