Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify remaining unknown chunks #261

Closed
fmatthew5876 opened this Issue Oct 12, 2018 · 11 comments

Comments

Projects
None yet
5 participants
@fmatthew5876
Copy link
Contributor

fmatthew5876 commented Oct 12, 2018

Does anyone have more information on BattleCommands 0x9? We call it 'unknown_09'.

These are the unknown ones still in fields.csv

#Structure Field Size Field? Type Index Default Value PersistIfDefault Is2k3 Comment
BattleCommands unknown_09 f Int32 0x09 -1 0 0 Integer
SaveSystem unknown_3d_music_fadeout f Int32 0x3D 0 0 0 music fade out? FIXME
SaveScreen unknown_2e_battleanim_active f Int32 0x2E 0 0 0 int - unsure FIXME
SavePartyLocation unknown_2f_overlap f Int32 0x2F 0 0 0 Boolean - Overlap? Almost the same as 0x33 (through). 0x2F represents that by MoveEvent the through mode has been activated. 0x33 represents whether the event is permeable straight and no matter which. This is actually only relevant events (not the hero) for example which can be set by an empty event page on translucent (0x33 = true) without having been set by a MoveEvent so (0x2F = false). FIXME
SavePartyLocation unknown_47_pause f Int32 0x47 0 0 0 used as a kind of pause flag during the event processing. Not quite sure what causes it. FIXME
SavePartyLocation unknown_4b_sprite_move f Int32 0x4B 0 0 0 Flag whether an event (the hero is also an event) in the current frame have any movement action has made.
SavePartyLocation unknown_6c_menu_calling f Int32 0x6C 0 0 0 flag which briefly is true if the player presses ESC and another place in the code is set to false when subsequently calls the menu.
SavePartyLocation unknown_7d_encounter_calling f Boolean 0x7D False 0 0 Similar to 0x6C - is used to signal a different piece of code that an Encounter to be triggered.
SaveVehicleLocation unknown_47_pause f Int32 0x47 0 0 0 used as a kind of pause flag during the event processing. Not quite sure what causes it. FIXME
SaveVehicleLocation unknown_4b_sprite_move f Int32 0x4B 0 0 0 Flag whether an event (the hero is also an event) in the current frame have any movement action has made.
SaveEventCommands unknown_15_subcommand_path_size f Int32 0x15 -1 0 0 size of the 0x16 vector: an array which stores the to be brought into an event code path FIXME
SaveEventCommands unknown_16_subcommand_path f Vector 0x16 1 0 byte Each indentation in the event code corresponds to an entry in the array. When a command such as e.g. Show Choice is achieved; stored in the array entry of the current level; which code path must accept the event. For example: if the player chooses the third entry is '3' (or maybe 2? not tested) stored there. When a 'Case XXX' is achieved command; it is checked whether the value is stored there; the value of the 'Case'-subcommand corresponds. Otherwise the block is skipped. If so then the block is executed and the stored value is set to 255 (probably a double protection if times; although that should never be more Case subcommands are with the same ID. FIXME
SaveEventData unknown_0b_escape f Int32 0x0B 0 0 0 Flag which is set before a fight if the event is canceled by the struggle for escape. FIXME
SaveEventData unknown_2a_time_left f Int32 0x2a 0 0 0 number of frames have to wait until the event continues. FIXME difference with 0x1F?
SaveMapEvent unknown_2f_overlap f Int32 0x2F -1 0 0 Boolean - Overlap? Almost the same as 0x33 (through). 0x2F represents that by MoveEvent the through mode has been activated. 0x33 represents whether the event is permeable straight and no matter which. This is actually only relevant events (not the hero) for example which can be set by an empty event page on translucent (0x33 = true) without having been set by a MoveEvent so (0x2F = false). FIXME
SaveMapEvent unknown_47_pause f Int32 0x47 0 0 0 used as a kind of pause flag during the event processing. Not quite sure what causes it. FIXME
SaveMapEvent unknown_4b_sprite_move f Int32 0x4B -1 0 0 Flag whether an event (the hero is also an event) in the current frame have any movement action has made.
@CherryDT

This comment has been minimized.

Copy link

CherryDT commented Oct 12, 2018

#Structure Field Size Field? Type Index Default Value PersistIfDefault Is2k3 Comment
BattleCommands unused_display_normal_parameters f Boolean 0x09 True 0 1 Unused hidden checkbox "Display normal parameters" in RPG2003's battle settings tab
SaveSystem music_stopping f Boolean 0x3D False 0 0 Music is being faded out or had been stopped ("Play music" with the same music as currently playing will restart the music when this flag is set)
SaveScreen battleanim_active f Boolean 0x2E False 0 0 There is currently a battle animation playing
SavePartyLocation route_through f Boolean 0x2F False 0 0 Whether the move route (MoveEvent or defined route) activated through mode. Almost the same as 0x33 (through). 0x2F represents that by MoveEvent the through mode has been activated, but 0x33 is what's actually checked for collisions. In several cases, 0x33 will be changed to indicate a condition in which an event or the hero is in through mode through other means than a MoveEvent, which can be: an event with an empty page being activated; player pressing Ctrl in test play; hero entering or exiting a vehicle (only very briefly)
SavePartyLocation pause f Boolean 0x47 False 0 0 Indicates paused movement for an event, set while the player is "talking" to the event so that it won't run away (not relevant for hero themselves)
SavePartyLocation processed f Boolean 0x4B False 0 0 Flag whether an event (the hero is also an event) in the current frame processed their movement actions (may also be none). This is required because events are asked every frame to initiate their next movement step if required, but not necessarily in order, because checking passability for an event trying to move onto another tile will trigger any event's movement initiation which is on the target tile (because this way the target event may move away, allowing the other event to move to that tile). This flag ensures that every event processes their possible movements only once per frame even if it was already asked to do so "out of order" as part of another event's movement initiation.
SavePartyLocation menu_calling f Boolean 0x6C False 0 0 Flag which briefly is true if the player presses ESC. At the right place in handling each frame's activities for the player, the code checks whether this flag is set and calls the menu, however there are several conditions which would cancel this flag and instead process another "higher-priority" action, such as when an encounter takes place during the same frame.
SavePartyLocation encounter_calling f Boolean 0x7D False 0 0 Similar to 0x6C - is used to signal a different piece of code that an encounter is to be triggered, which may be cancelled by other conditions such as the player starting to interact with an event during the same frame.
SaveVehicleLocation pause f Boolean 0x47 False 0 0 Indicates paused movement for an event, set while the player is "talking" to the event so that it won't run away (not relevant for vehicles themselves)
SaveVehicleLocation processed f Boolean 0x4B False 0 0 Flag whether an event (the hero is also an event) in the current frame processed their movement actions (may also be none). This is required because events are asked every frame to initiate their next movement step if required, but not necessarily in order, because checking passability for an event trying to move onto another tile will trigger any event's movement initiation which is on the target tile (because this way the target event may move away, allowing the other event to move to that tile). This flag ensures that every event processes their possible movements only once per frame even if it was already asked to do so "out of order" as part of another event's movement initiation.
SaveEventCommands subcommand_path_size f Int32 0x15 0 0 0 size of the 0x16 vector - indention level
SaveEventCommands subcommand_path f Vector 0x16 1 0 byte For each indention level in the script, an ID is stored there which corresponds to the branch to take in case a command allows multiple branches. For example, the "Show Choice" command would write the result of the choice (for example "2" for the third item) into the current indention level's entry in this array, and the script processor would later look for the "Case" subcommand with the corresponding ID, if any, and jump to that one (if none found, it would jump to the "End Case" subcommand). Once the jump is executed, the ID is set to 255 (probably a protection mechanism even though there should normally not be multiple subcommands with the same ID).
SaveEventData abort_on_escape f Boolean 0x0B False 0 0 Flag which is set before a fight if the EnemyEncounter event command had battle_escape_mode set to 1 (abort event on escape). After the fight, the interpreter checks if the battle result was an escape and this flag was set and abort the event in that case.
SaveEventData unused_wait_for_key_or_enter f Boolean 0x2a False 0 1 If enabled, an event waits for either the confirmation key to be pressed or one of the keys defined by KeyInputProc before continuing. This flag seems to be unused though since it is never written to (keyinput_wait is used instead).
SaveMapEvent route_through f Boolean 0x2F False 0 0 Whether the move route (MoveEvent or defined route) activated through mode. Almost the same as 0x33 (through). 0x2F represents that by MoveEvent the through mode has been activated, but 0x33 is what's actually checked for collisions. In several cases, 0x33 will be changed to indicate a condition in which an event or the hero is in through mode through other means than a MoveEvent, which can be: an event with an empty page being activated; player pressing Ctrl in test play; hero entering or exiting a vehicle (only very briefly)
SaveMapEvent pause f Boolean 0x47 False 0 0 Indicates paused movement for an event, set while the player is "talking" to the event so that it won't run away
SaveMapEvent processed f Boolean 0x4B False 0 0 Flag whether an event (the hero is also an event) in the current frame processed their movement actions (may also be none). This is required because events are asked every frame to initiate their next movement step if required, but not necessarily in order, because checking passability for an event trying to move onto another tile will trigger any event's movement initiation which is on the target tile (because this way the target event may move away, allowing the other event to move to that tile). This flag ensures that every event processes their possible movements only once per frame even if it was already asked to do so "out of order" as part of another event's movement initiation.
@CherryDT

This comment has been minimized.

Copy link

CherryDT commented Oct 12, 2018

While we are at it, I looked at the other FIXME entries in fields.csv:

MapInfo.encounter_steps doesn't have special behavior for 0 as far as I can see. However, SaveMapInfo.encounter_rate does have a special behavior (why is it named differently by the way?) - SaveMapInfo.encounter_rate==-1 means that MapInfo.encounter_steps is used instead. And a value of 0 means encounters are disabled (which is useful because this value can be modified by event commands).

SaveScreen.battleanim_frame is just a simple frame counter (in game frames, i.e. half animation frames).

SaveVehicleLocation.vehicle is indeed just the ship ID, same enum as SavePartyLocation_VehicleType.

And for SaveMapEvent.pending I commented in #174 (data type is just boolean, but same as a few other booleans it may be possible that someone changed the datatype to sneak in additional bytes of data)

In general, wouldn't it make more sense to define a class inheritance instead of duplicating the common chunks for events, party and vehicles? I mean, event, party and vehicle simply inherit from the "character" class in RM, and their save/load behavior has differences above a certain chunk ID. As far as I understand, RM normally uses a different "hundreds" digit for derived classes.

Chunk IDs are decimal numbers with XYZ where X is the inheritance level, 0 for base class, 1 for derived class, etc., Y is the "category", Z is a counter. For example, looking at party: 0XX would be for the base "character" class and the same in events, party and vehicle. 1XX are different. And looking at the party, we see for example that ship and movement related things are in chunks 10X, map position/scrolling related things in chunks 11X, encounter-related things in chunks 12X (except for 121, not sure why they put it there) and magic numbers in 13X. (Event command IDs use a similar numbering scheme.) I guess it's too big a change for you, but I would find it nice to have the decimal IDs here, both for file chunks and for event command IDs - that would make their meaning more apparent.

@carstene1ns

This comment has been minimized.

Copy link
Member

carstene1ns commented Oct 12, 2018

@CherryDT First of all: Thank you for all that information!

Changing the IDs from hex to decimal should be easy to do, as this is just a convention.
If it makes understanding such things easier, I will always vote for it.
However, the inheritance you describe would require a refactor.
(Doing such things is okay, at least the API of liblcf is by no means stable yet)

@CherryDT

This comment has been minimized.

Copy link

CherryDT commented Oct 12, 2018

I see - for the inheritance, it's not worth it I guess, it's just that duplicate things make me feel uneasy when it comes to programming :-)

@carstene1ns

This comment has been minimized.

Copy link
Member

carstene1ns commented Oct 12, 2018

We already have a generator, because we do not like to duplicate stuff as well 👍

@fmatthew5876

This comment has been minimized.

Copy link
Contributor Author

fmatthew5876 commented Oct 12, 2018

@CherryDT First of all: Thank you for all that information!

I concur, this is fantastic.

@CherryDT , regarding inheritance what about the way RPG_RT decides whether or not to write a defaulted chunk? If B inherits from A, will all of the common fields follow the same defaulting rules in all cases when LDB/LMT/LMU/LSD are written?

I tried many ways in #242 to find a pattern but I couldn't so I just put a presentifdefault flag in fields.csv and manually tagged every field I could figure out by diffing game binaries. It would be much better if there was a simple set of rules dictating whether or not a chunk is written out we could follow instead of this brittle approach of manually tagging everything. Do you have any knowledge of how RPG_RT.exe does this?

Despite having the generator, some inheritance could be useful. We would have less code and less types, with logic being reused and shared. Lets copies of identical things with different names in general means less bugs and more composability and code reuse.

For the Chunk ID, both decimal and hex are useful. When I'm looking at a binary in a diff program like Beyond Compare it's really useful to just compare hex to hex without having to convert in a calculator. If anything, maybe we should have both. One as the value in the code and the other in a comment right next to it.

While we are at it, I looked at the other FIXME entries in fields.csv:

After #242 I want to look at perfect copying of LSD files. As part of that process, I'd like to deep dive and make sure our saving and loading is being done correctly. If you see anything else wrong in Player/liblcf would be happy to know so I can fix it.

@CherryDT

This comment has been minimized.

Copy link

CherryDT commented Oct 12, 2018

Summary of my monologue in chat:

vectors and structs are always written, as are things that are enums in the RM code (the latter also have a fixed size and are not var-len integers). Strings may or may not have an empty string as default (this is indeed a flag).

@fmatthew5876

This comment has been minimized.

Copy link
Contributor Author

fmatthew5876 commented Oct 13, 2018

I noticed the enum and vector thing when I started the #242 and first tried a type based approach. But then I found some fields that don't obey the rules. Probably some things we consider "enums" are just ints in RPG_RT etc..

At a high level, I think its better to optimize our data structures for C++ and Player with hacks on the read/write side for compatibility. The alternative is designing our data to be exactly like RPG_RT in Delphi and then having hacks everywhere to make it run well in modern C++ with Player and Editor.

@fmatthew5876

This comment has been minimized.

Copy link
Contributor Author

fmatthew5876 commented Oct 19, 2018

Terrain::Flags and SavePicture::Flags dissapear when defaulted. TroopPageCondition::Flags and EventPageCondition::Flags are present when defaulted.

We model these 4 as the same kind of type. How does RPG_RT make a logical distinction between these 2 pairs of flags?

@CherryDT

This comment has been minimized.

Copy link

CherryDT commented Oct 19, 2018

Event/TroopPageCondition's flags are a Delphi set type which cannot be converted to a number and is hence stored as generic type with fixed size (and without default), not as integer.

The other two are semantically the same, but the code was written by different people and those people used integers to store the bitfield instead of a set type: Terrain flags by the 2k3 developers and picture flags by me.

@Ghabry

This comment has been minimized.

Copy link
Member

Ghabry commented Oct 30, 2018

this is also resolved I think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.