Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Games with altered file extensions #1920

Closed
Ghabry opened this issue Oct 6, 2019 · 5 comments · Fixed by #2271
Closed

Games with altered file extensions #1920

Ghabry opened this issue Oct 6, 2019 · 5 comments · Fixed by #2271

Comments

@Ghabry
Copy link
Member

Ghabry commented Oct 6, 2019

A few games, e.g. Laxius Power use modified file extensions.

Player will fail to detect such games because RPG_RT.ldb is missing.
No idea how to support this.

File extension changes:
Rpg_rt.ldb to Rpg_rt.set
Rpg_rt.lmt to Rpg_rt.lxb
and all the MapXXXX.lmu to MapXXXX.ont

@sorlok
Copy link
Contributor

sorlok commented Nov 5, 2019

So I'd like to try fixing this. There's basically 3 things that need to be done:

  1. Detect that a folder might have renamed game files.
  2. Figure out which files were renamed.
  3. Fix any internal string renames (things like "LcfMapUnit" are sometimes mangled on purpose).

Note that for all 3 steps, I'd like to avoid reading bytes from RPG_RT.exe directly if possible.

(1) is easy: count the number of "RPG_RT." files (should be 3) and "MapXXXX." files (should be >0).

For (2), we can assume from (1) what the Map extension is. Of the 3 RPG_RT.* files, 1 is ".ini", and the larger of the remaining 2 is almost certainly the database. We could also try parsing the guessed "LMT" file and seeing if it passes, but this seems like overkill. Finally, I'd like to add an option to read the custom extensions from metadata (since we have the RPG_RT.* checksums).

For (3), I think we'll need to just disable the "LcfXXX" checks inside the liblcf parser whenever a custom file extension is in use. Alternatively, if we used metadata for (2), we can also retrieve the "LcfXXX" replacements from metadata, which is a little more reliable.

This approach should support custom extensions with minimal fuss. Let me know if this seems like a good approach and I'll start on a prototype.

@Ghabry Ghabry added this to the 0.6.2 milestone Nov 5, 2019
@Ghabry
Copy link
Member Author

Ghabry commented Nov 5, 2019

Thanks for working on this :)

Let me know if this seems like a good approach and I'll start on a prototype.

Yes, your approach sounds good. The main problem I saw was that reading of files is necessary which gives an unacceptable slowdown (especially in the Game browser which goes through all folders) but your heuristic works without this.


This issue can be split in two categories:

Main issue:

  • Support loading of "folders" with altered file extensions

Secondary issues:

I would consider this resolved (and open a new issue) when the Main issue is fixed but you can decide if you want to fix all of them.


(1) is easy: count the number of "RPG_RT." files (should be 3) and "MapXXXX." files (should be >0).

Better check for 2 as some games have no .ini file.

For (2), we can assume from (1) what the Map extension is. Of the 3 RPG_RT.* files, 1 is ".ini", and the larger of the remaining 2 is almost certainly the database.

Okay lets assume for now that huge = LDB, smaller = LMT, smallest = INI.
Good thing is: This heuristic will be only needed when the extensions are altered, so the false positive rate should be very low. (and when the heuristic fails we can expect a bug report).

For (3), I think we'll need to just disable the "LcfXXX" checks inside the liblcf parser

These checks are already skipped. The parser only checks the string length (there is a huge amount of games with hex edited headers).

@Ghabry Ghabry removed this from the 0.6.2 milestone Apr 9, 2020
@sorlok
Copy link
Contributor

sorlok commented Jul 23, 2020

Good evening,

PR #2271 has my first draft at a fix for this. Apologies for dropping off the face of the earth; I broke some bones in an accident and was unable to do computer work for a while.

It's been a while since I started this, so please let me know any feedback for my fix (and then I will rebase it). Tested with Laxius Power, Battle for Genesis, and a private project that renames LMU files.

Thanks!

@sorlok
Copy link
Contributor

sorlok commented Jul 24, 2020

As of the latest commits, I've confirmed PR #2271 fixes all 3 issues identified above (loading from a directory, using the game browser, and using the game browser on Android).

I'd still like some feedback, since my programming's a bit rusty. (Also, my email claims there was a comment on regexes, but it seems to have disappeared.)

@sorlok
Copy link
Contributor

sorlok commented Jul 26, 2020

I've made additional fixes as suggested, and rebased+squashed PR #2271 . I think I'm done with things on my side.

For those wishing to scan their own project folders for non-standard games, the following script should prove useful:
https://drive.google.com/file/d/1iv2YHQ_DFXRWgixeojaWUbgxZkzJ7ODl/view?usp=sharing

@fdelapena fdelapena linked a pull request Aug 23, 2020 that will close this issue
@Ghabry Ghabry added this to the 0.6.3 milestone Sep 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging a pull request may close this issue.

3 participants