-
Notifications
You must be signed in to change notification settings - Fork 1
File Format documentation
A GR2 file is a simple key-value binary file format, similar to what a binary json might look like. The key-value is referenced as GRN (Granny1) nodes inside the DLL. The key-value contains all the relevant information of the file, including vertices, exporter info and so on.
The GRN nodes are then splitted and compressed into different chunk of data, and specific operations are applied to make sure the file is portable between multiple endianess, architectures or platforms.
Please keep in mind that the following document contains the information of a basic Gr2 format (Synced up to Granny 2.12.0.2 found in Metin2 steam client), some games could contain custom nodes or custom information as the format was meant to be extensible by purchasing a source license.
A file contains the following information: [Header] [File Info] [Sector information] [Data]
The header contains the basic information about the file (such as the magic which identifies if the file is encoded as Big Endian or Little Endian or if it uses 64-bit pointers) The header also contains two important piece of informations, the File Format, which defines the version of the Granny2 structures and the Tag, which defines the version of the content of the Gr2 (more on that later). THe header also contains the information about the important sectors, the type sector and the root sector.
The granny2 file is divided in different sectors (usually 8) which contains the actual data of the file, such as the vertices or the exporter information. This files contains the size of the sectors, it's compression type used, and their position relative to the file. It also contains information about which marshalling and fixup data should be applied to the content of that sector.
This sector contains the information about the node names, their childrens, the type of node they are and if they are an array type or not. There are 22 basic nodes contained inside the GR2 library. The TAG information on the header contains the version of the types contained inside the root sector.
The original DLL does not dynamically parse the types, it only relies on using this sector when there is a tag mismatch to convert the information and data to the latest version included in this DLL.
This sector contains the raw value of the data expressed in the type sector. Remember that there is no direct association in the type sector to where the data is, so you have to parse the data from the type sector assuming everything in the type sector is right.
The original DLL parses this sector right away (assuming the file tag is the same as the DLL tag, or the file has been converted to use this tag) without performing any checks on the type sector, it relies on an information type array to parse the information.
The data might be the actual sector data, fixup data or marshalling data. The data is expressed in binary format. A library should be able to parse all the 8 sectors of the file, decompress them, apply specific type marshalling or fixup data and rebuild the nodes.
You can view this file to see the header structures of a GR2 file.
The file is composed as such:
[THeader] [TFileInfo] [TSector * number of sectors] [Data]
You can decompose data by looking at the sector information, for example sector 1 might contain marshalling data. You can move your file pointer to the position pointed by the sector and start parsing the data.
The marshalling data is composed as such: [TMarshallData * number of marshals]
The fixup data is composed as such: [TFixUpData * number of fixups]
After parsing and decompressing the sector data, two following operations will be executed, the Marshalling operation and the Fixup operation, explained below. Once this two operations are done, the file is ready to be properly parsed.
A granny2 file could be operate as little endian or big endian, the big endian platform is particulary usefull for console platforms like PS3. Based from the magic, the library detects if the file was exported as big endian or little endian, then it performs a simple check of the current CPU endianess to determine if we are in an endianess mismatch situation.
When this situation applies, the library swaps the bytes it reads to match the one of the current CPU, it would also perform some sector data swapping based on the Stop0 and Stop1. (You can check this file to see all the theorical byte swapping).
In this situation, a special step of the file is performed, a step that would be ignored in a case where both the CPU and the file are on the same endianess, this step is known as the Marshalling. The Marshalling job is to fix the Gr2 node information to contain reversed node information that would match the structure of the file.
- get the node types (this will be reversed type nodes)
- byteswap the node type info to get it's width + nodetype (gr2 byteswap seems to ignore the last 16 bytes unk[4])
- get the size of all nodes (array includes)
- multiply element size with the array size
Based from the type of the node, one of this case should apply:
- if the type is inline, restart from step1 using the childOffset (swapped) as the start point
- if the type is not inline and the element swap size is 4, use Swap1 to all typearray size (array size * element size)
- if the type is not inline and the element swap size is not 4, use Swap2 to all typearray size (array size * element size)
You could have noticed by looking at this file, which contains all the GRN node informations and types, that the [TElementInfo] that a single GRN node contains a type, a name and some childrens.
This information (which is already formatted in opengrn), is defined in the file as the following structure: struct NodeInfo { uint32_t type; ptrdiff_t name_offset; ptrdiff_t children_offset; uint32_t array_size; uint8_t unknown[12]; ptrdiff_t unknown2; };
Remember that ptrdiff_t means 4 bytes in a 32-bit GR2 file, but 8 bytes in a 64-bit GR2 file. You can notice the two offsets which should be mapped as pointers that contains the actual data, but pointers are not portable between code, so reading a file and trying to get (for example) the name from this node information would lead to an invalid pointer reference and to a crash shortly after. This is where the FixUp operation comes in, it's job is to map data of a sector to a specific pointer. After the fixup operation, the offset value would have been replaced by pointers contained inside our own memory, which would lead into making the parsing correct.
This offsets is also behind the reason of why a Granny2 can operate a 32-bit file or 64-bit, a 32-bit file would contain 32-bit pointers, a 64-bit file would contain 64-bit pointers. The granny2 also has a specific type, called Reference type, where pointers are being used (and so they are resolved by the fixup as well).
[Note: I have not looked on how the library deals when having bit mismatch, as a way to resolve this issue I would suggest a virtual pointer table]
- Read the fixup information
- Get the offset of the target information
- Copy the pointer offset into the source information, effectively fixing the bad pointer