Repository navigation
How to detect an FM's game type
Here's how to detect the game type of both zipped FMs and uncompressed FMs in folders. If you can't stream your archive format to memory, then you can extract the relevant files to a temp folder first and thereafter follow the uncompressed-in-folders instructions.
These instructions are for if you want maximum performance. If you don't care about that, then things get a lot simpler: Just search any used .mis file (see below) for RopeyArrow (Thief II) and MAPPARAM (System Shock 2). If you don't find either, it's Thief TDP/Gold, and you're done. But if you want performance, read on.
But before that, a couple of notes:
First, I won't get into how to get maximum performance from 7z files here, because it's extremely complicated and so I'm just going to call it "outside the scope of this guide" and move on.
Second, the big performance bottleneck with compressed FMs is that a .mis file's table of contents is located at the end of the file, and is therefore virtually useless as a shortcut to jump to a particular chunk: if you're at the table of contents, then you've decompressed through nearly the entire file anyway, so you're saving next to no time over a full-file brute-force search. This is why the compressed detection code paths rely heavily on buffered block-by-block brute-force searches: if you find what you're looking for long before you hit the table of contents, you've saved a ton of time.
Now, on with the guide.
First, we need to do some prep.
Create two lists of filenames. Call one misFiles and the other usedMisFiles.
Next, add all .mis filenames in your FM's base directory to misFiles. Now look for a missflag.str file anywhere in \strings or its subdirectories.
If you find a missflag.str file, then check every filename in misFiles against the lines in missflag.str in the following way:
Pseudocode:
for each misFile in misFiles
where misFile is of format 'missxx.mis' // xx is some number of decimal digits
and missFlagFile contains a line of format 'miss_xx: "[value]"'
where xx matches the xx in the missxx.mis file name // For example: 'miss20.mis' and 'miss_20: "[value]"'
and "[value]" is anything other than "skip"
{
add misFile to usedMisFiles
}
For example, if misFiles contains:
miss19.mis
miss20.mis
miss21.mis
miss22.mis
and missflag.str looks like this:
miss_1: "skip"
miss_2: "skip"
miss_3: "skip"
miss_4: "skip"
miss_5: "skip"
miss_6: "skip"
miss_7: "skip"
miss_8: "skip"
miss_9: "skip"
miss_10: "skip"
miss_11: "skip"
miss_12: "skip"
miss_13: "skip"
miss_14: "skip"
miss_15: "skip"
miss_16: "skip"
miss_17: "skip"
miss_18: "skip"
miss_19: "skip"
miss_20: ""
miss_21: "no_briefing, no_loadout"
miss_22: "no_briefing, no_loadout, end"
then usedMisFiles should contain:
miss20.mis
miss21.mis
miss22.mis
but should NOT contain:
miss19.mis
If you didn't find a missflag.str, then add all filenames in misFiles to usedMisFiles.
Now create a list called gamFiles and add all .gam filenames in your FM's base directory to it. It's okay if there are none.
From usedMisFiles, find the .mis file with the smallest filesize. Call it misFile.
From gamFiles (if any), find the .gam file with the smallest filesize. Call it gamFile.
Okay, that's the prep. Now we can start the detection.
In misFile, check for the byte sequence SKYOBJVAR at byte location 772. If it exists at that location, the FM is for Thief II. Note that checking for SKYOBJVAR at any other location is NOT a reliable way to detect game type! This is in fact why DarkLoader misdetects all NewDark FMs as Thief II. The SKYOBJVAR-at-772 check is merely a shortcut for performance: only OldDark Thief II FMs have it at this location.
If you have not detected Thief II, then check for the byte sequence MAPPARAM at locations 696 (OldDark) and 916 (NewDark). If you have a .gam file, you can also check it for GAMEPARAM at locations 1194 and 1566. (This may give you better performance if you have a solid FM archive (eg. 7z) and you want to minimize extraction time by extracting a .gam file that is earlier in the solid block than the .mis file.) If the appropriate byte sequence is found at any of these locations, the FM is for System Shock 2.
If you have not detected System Shock 2, then you should now check for the NewDark 1.28+ Mission Game Parameter byte. This is the 14th byte after the DARKMISS chunk's header (chunk headers are 24 bytes: 12 bytes for the name, then 12 more bytes of cruft that doesn't concern us here).
Example of a NewDark >= 1.28 FM:
DARKMISS chunk length is 14.
DARKMISS....................miss20....AIPATHVAR
^ ^ ^
| | |
Chunk header start | This is the location of the Mission Game Parameter byte.
Chunk header end
Example of a NewDark <= 1.27 FM:
DARKMISS chunk length is 13.
DARKMISS....................miss20...AIPATHVAR
^ ^ ^
| | |
Chunk header start | This is the first byte of the next chunk name - no Mission Game Parameter exists.
Chunk header end
There are two ways to find the DARKMISS chunk. If you have random-access to misFile (ie. it's uncompressed), you can do the following (pseudocode):
// All integers are stored and read as little-endian.
let tableOfContentsIndex = read_uint32()
seek(tableOfContentsIndex)
let tableOfContentsEntriesCount = read_uint32()
let buffer = new byte[12]
for i = 0; i < tableOfContentsEntriesCount; i++
{
// This is the chunk name, zero-padded at the end if less than 12 bytes.
let chunkName = read(buffer, 12)
// This is the index of the chunk from the start of the file.
let chunkIndex = read_uint32()
// This is the length of the chunk (a chunk's 24-byte header is NOT counted in the length).
let chunkLength = read_uint32()
// \0 means null byte.
if chunkName == 'DARKMISS\0\0\0\0'
{
if chunkLength == 13
{
// FM is NewDark 1.27 or earlier and does not have a Mission Game Parameter byte.
// Abort the check.
break
}
else // Chunk length is 14 or greater
{
// FM is NewDark 1.28 or later and has a Mission Game Parameter byte.
// Proceed with checking the byte.
seek(chunkIndex)
let chunkName_shouldBeDARKMISS = read(buffer, 12)
if chunkName_shouldBeDARKMISS != 'DARKMISS\0\0\0\0'
{
// Chunk names didn't match; corrupted .mis file; abort the check.
break
}
else
{
// 12 for the second half of the header, 14 to get to the byte
seek_forward_by(12 + 14)
// (Do byte check here)
break
}
}
}
}
If misFile is compressed and you don't have random access, you can simply check byte location 612 for the sequence DARKMISS, and if found, check the Mission Game Parameter byte from there. All NewDark T1 and T2 FMs at the time of writing have the DARKMISS chunk at location 612. If this were to change in the future, the check would fail and you would simply fall back to one of the slower checks, so everything would still work.
Also, in this case, you won't have read the table of contents and therefore won't know the length of the chunk, and so if the Mission Game Parameter byte does not exist, you'll end up reading the first byte of the next chunk header. However, this works out fine, as chunk headers always start with a printable ASCII character, which will never accidentally map to a valid Mission Game Parameter value.
Now that you have the byte in question, you can check it against these values:
0x01 - Any
0x02 - Thief TDP or Thief Gold
0x03 - Thief II
Note that there is no value for System Shock 2, and SS2 FMs don't have the Mission Game Parameter byte at all. This is because there are already other ways of detecting SS2.
The useful values for game detection purposes are therefore 0x02 and 0x03. If you see any other value, then carry on to the checks below.
If gamFile exists and is smaller than misFile (it usually will be), search it for the byte sequence RopeyArrow. If found, the FM is for Thief II. If you have random access, you can search just the P$SymName chunk. Otherwise, decompress through the file in buffered blocks (making sure to implement checking for matches on block boundaries) and search each block for the sequence.
If gamFile does not exist or misFile is smaller, then do the same as above for misFile, except the chunk to search in the random-access case is OBJ_MAP.
At this point, if all checks have failed then the mission is for Thief TDP/Gold.
Since System Shock 2 does not have a fallback path like RopeyArrow for T1/T2, it's possible for the FM to be misdetected as Thief Gold if the locations of MAPPARAM or GAMEPARAM were to change in a future NewDark version. To solve this, you could either search the entire .mis or .gam file for the appropriate byte sequence (which would mean all OldDark and NewDark <=1.27 non-T2 FMs would get their files fully searched, slowing down detection drastically) or else you could use use what I call "fingerprinting" as a fallback/assist. System Shock 2 FMs have several folders and file names that don't exist in T1/T2 FMs, and its .mis files are named differently as well - not missxx.mis but for example earth.mis (a stock name) or ruins.mis (named after the FM's title). You can "fingerprint" the FM as a likely SS2 mission from its files and folders, and then if the detection has come back as Thief TDP/Gold but you've fingerprinted the FM as SS2, you can confirm with a full search of the .mis or .gam file for the appropriate byte sequence.
AngelLoader fingerprints SS2 as follows:
If any files with any of the following names exist in any subfolders:
usemsg.str
savename.str
objlooks.st
OBJSHORT.str
or if any folders with any of the following names exist in the base directory:
snd2
cutscenes
or if any .mis files have any of the following names:
command1.mis
command2.mis
earth.mis
eng1.mis
eng2.mis
hydro1.mis
hydro2.mis
hydro3.mis
many.mis
medsci1.mis
medsci2.mis
ops1.mis
ops2.mis
ops3.mis
ops4.mis
rec1.mis
rec2.mis
rec3.mis
rick1.mis
rick2.mis
rick3.mis
shodan.mis
station.mis
then we fingerprint the FM as SS2.
First, check if your zip or dir contains the path "Content/T3/Maps/" and if that directory contains one or more files with any of the following extensions: .ibt, .cbt, .gmp, .ned, or .unr. If so, the FM is for Thief: Deadly Shadows.
No detection is needed - The Dark Mod keeps its FMs inside its game folder, unlike the other games where the FM archive files are usually located somewhere else. So you know an FM is for the Dark Mod simply by virtue of its location on disk. However, if you need to detect a Dark Mod FM out of its usual folder context, you can again easily differentiate it from the other games by its file/folder names and structure.