Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect corrupted chunks and reset the parser #250

Merged
merged 1 commit into from Oct 27, 2018

Conversation

Projects
None yet
5 participants
@fmatthew5876
Copy link
Contributor

fmatthew5876 commented Oct 3, 2018

Towards addressing #249

This one successfully parses the Devil and Angel ldb. When I copy it, the chunk for the corrupted animation is different, but everything else in the database is a perfect match.

This and other games like like it should load and be playable now in Player.

This doesn't attempt to emulate how rm2k/3 interprets the corrupted chunk.

@fmatthew5876 fmatthew5876 force-pushed the fmatthew5876:ldb_safe branch 2 times, most recently from 7836588 to ce458cf Oct 3, 2018

@Ghabry

This comment has been minimized.

Copy link
Member

Ghabry commented Oct 3, 2018

Defect report for you:

2k
Der Sinn des Lebens (Demo 2)/
Warning: Corrupted Chunk 0x01 (size: 37, pos: 0x20211): cells : Read 79 bytes! Reseting...

Devil or Angel/
Warning: Corrupted Chunk 0x02 (size: 1, pos: 0x1b0f5): se : Read 138 bytes! Reseting...
Warning: Corrupted Chunk 0x04 (size: 2, pos: 0x1b0f8): flash_red : Read 1 bytes! Reseting...
Warning: Corrupted Chunk 0x08 (size: 2, pos: 0x1b12b): screen_shake : Read 1 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x1b3fb): se : Read 126 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x1bebb): se : Read 141 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x1ce24): se : Read 141 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x1dd8c): se : Read 192 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x1e71d): se : Read 157 bytes! Reseting...
Warning: Corrupted Chunk 0x01 (size: 4, pos: 0x1f052): frame : Read 1 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 9, pos: 0x1f058): se : Read 159 bytes! Reseting...
Warning: Corrupted Chunk 0x06 (size: 222, pos: 0x1b0e1): timings : Read 17157 bytes! Reseting...

Kroetenstuhl_V040/
Warning: Corrupted Chunk 0x04 (size: 5, pos: 0x22d42): y : Read 6 bytes! Reseting...

Krötenstuhl Saga (Demo 5a)/
Warning: Corrupted Chunk 0x04 (size: 5, pos: 0x20c01): y : Read 6 bytes! Reseting...

Rick Jones Hunters (Demo)/
Warning: Corrupted Chunk 0x06 (size: 190, pos: 0x299d1): timings : Read 554 bytes! Reseting...

2k3
Monster World RPG/
Warning: This header is not LcfDataBase and might not be a valid RPG2000 database.
Warning: Corrupted Chunk 0x02 (size: 1, pos: 0x37521a): se : Read 138 bytes! Reseting...
Warning: Corrupted Chunk 0x04 (size: 2, pos: 0x37521d): flash_red : Read 1 bytes! Reseting...
Warning: Corrupted Chunk 0x08 (size: 2, pos: 0x375250): screen_shake : Read 1 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x375520): se : Read 126 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x375fe0): se : Read 141 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x376f49): se : Read 141 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x377eb1): se : Read 192 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 3, pos: 0x378842): se : Read 157 bytes! Reseting...
Warning: Corrupted Chunk 0x01 (size: 4, pos: 0x379177): frame : Read 1 bytes! Reseting...
Warning: Corrupted Chunk 0x02 (size: 9, pos: 0x37917d): se : Read 159 bytes! Reseting...
Warning: Corrupted Chunk 0x06 (size: 222, pos: 0x375206): timings : Read 17157 bytes! Reseting...

Seven Blades - Darkness (v0.44)/
Warning: Corrupted Chunk 0x01 (size: 103, pos: 0x56534): cells : Read 100 bytes! Reseting...
Warning: Corrupted Chunk 0x0c (size: 2850, pos: 0x55e40): frames : Read 1904 bytes! Reseting...

Which means the problem is in general no big deal.

(Marked 0.6.0 because I'm not sure if this fix will break some savegames written by EasyRPG, better don't risk anything :) )

@fmatthew5876

This comment has been minimized.

Copy link
Contributor Author

fmatthew5876 commented Oct 3, 2018

Matches my results. Thanks!

@Ghabry Ghabry added this to the 0.6.0 milestone Oct 3, 2018

@CherryDT

This comment has been minimized.

Copy link

CherryDT commented Oct 3, 2018

The RPG Maker does the same.

I found it interesting that there were two games (Devil or Angel and Monster Hunter RPG) with the exact same corruption, but in total different locations in the file. I looked at it and saw that they are both based on the same template, which I believe to be an obscure Italian-translated RM2k default database (I guess Monster World was converted from 2k to 2k3).

My guess is that in the world of corrupted downloads 15 years ago, this version was corrupted once and then it spread in some Italian-translated RM2k download...

The corruption is a 00 which should have been a 03 - that fixes it.
image

I also checked Rick Jones Hunters and there is a single bitflip at a different place, but also inside animation timings (a 09 which should have been a 01):
image

I do wonder though why this seems to happen almost exclusively in animation timing data. I'm worried that there is some kind of bug in RM (like a buffer overflow or something) which can cause these issues infrequently...

Now that's nothing liblcf can do of course, I just found it interesting nonetheless.

@Ghabry

This comment has been minimized.

Copy link
Member

Ghabry commented Oct 3, 2018

I find it interesting that RPG_RT is able to read such a corruption but when you have one extra trailing 0-byte at the end of a savegame you get "Stream Read Error" m(

@fmatthew5876 fmatthew5876 force-pushed the fmatthew5876:ldb_safe branch from ce458cf to 979460c Oct 11, 2018

@Ghabry Ghabry merged commit d9d8e90 into EasyRPG:master Oct 27, 2018

5 checks passed

GCW0 Build finished.
Details
GNU/Linux Build finished.
Details
OSX Build finished.
Details
Windows Build finished.
Details
web Build finished.
Details

@fmatthew5876 fmatthew5876 deleted the fmatthew5876:ldb_safe branch Oct 27, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.