[dev] Plugin Format Introduction
This document serves as a crash course through the plugin format and how Wrye Bash models it.
Plugin files for Creation Engine games are composed of records. As
stated in the UESP wiki
Records generally correspond to objects (e.g., a creature, a game setting, a dialog entry), with the fine details of the object (e.g., health of a creature, a dialog entry test) being handled by the fields of the record.
A record is build from a fixed size record_header and variable size
data, composed of one or more subrecords and / or fields, following
a particular layout per record type.
A subrecord is just a collection of one or more subrecords and / or fields, while a field is a basic unit of information - for example, a string, a FormID, an unsigned 32-bit integer or a set of flags.
The record_header
contains a 4 bytes identifier, which decoded to ascii gives the record
signature which is also referred to as the record type. Since type
is a reserved python keyword, and somewhat vague, both in this discussion and the code we
will be using signature to designate the record type. Different
signatures correspond to different records and different in-file
layout. The picture is further complicated with subrecords that also
have a subrecord_header and signature - not all signatures can
appear anywhere in the file and especially subrecords' signatures may
change layout format depending on the parent record.
For example, WTHR identifies a weather record, while DNAM can refer to a number of different fields and subrecords, depending on the record or even subrecord it's in - try searching 'DNAM' in PyCharm).
Records themselves are organized into groups (per record signature),
with the exception of the TES4 record (confusingly referred to as
"header" or "plugin header") which is unique and it's the first binary
block of the file, containing info for the plugin size, masters
etc. After the TES4 record (or TES3 for morrowind) comes a succession of
top level groups, (usually) one group per top level group signature
(not all record signatures can appear in a top level group). A top level
group is composed of a group_header with the same size but
different information than the record's header. Most notably this
group_header contains the top group signature, that is the
signature of the records to follow. Most top level groups are just a
succession of same signature records - with the exception of
CELL, DIAL and WRLD, which contain another layer of groups
(often called subgroups, again using special group_headers)
Headers: are implemented in RecordHeader and its subclasses, GrupHeader and TopGrupHeader - more to follow
Groups: Most top-level groups are implemented in MobObjects. The exceptions are obviously CELL (implemented in MobICells), DIAL (implemented in MobDials) and WRLD (implemented in MobWorlds). Subgroups are implemented as MobCell, MobWorld and MobDial.
Records: Are called Mre*, e.g. MreAchr, MreRegn, MreWthr, etc. They define a MelSet, which then includes
the subrecord and field definitions for the record. Remember that the main "header" of a mod file is the single top level
record that comes not in a group and it's the first record in the file. In Wrye Bash, it is implemented in
game.*.records.MreHeader which inherit from MreHeaderBase.
Subrecords: Wrye Bash generally calls these Mel* (if they're common and / or need special loading / writing) or defines them inline as a MelStruct (if their structure is known) or as a MelBase (if their structure is unknown / unimportant).
Fields: Strings and FormIDs have predictable names like MelString or MelFid, while integers and floats are loaded inside MelStructs (e.g. MelStruct('FNAM','f','fadevalue',),, from skyrim.records.MreLigh).
Flags are an interesting case. Wrye Bash uses a class called bolt.Flags to represent them. This class will read data given to it in the form of bits and interpret it as a series of (potentially named) binary masks with doubling values.
For example, this definition (from oblivion.records.MreDoor):
Flags.getNames('oblivionGate', 'automatic', 'hidden', 'minimalUse')would result in a mapping like this:
| Binary Mask | Flag Name |
|---|---|
| 1 | oblivionGate |
| 10 | automatic |
| 100 | hidden |
| 1000 | minimalUse |
However, it is also possible to leave out flag values. This is useful when some flag values are unknown or unused.
For example, consider this flag definition from oblivion.records.MreArmo:
Flags.getNames((16, 'hideRings'),
(17, 'hideAmulet'),
(22, 'notPlayable'),
(23, 'heavyArmor'))Here, the mapping will look like this:
| Binary Mask | Flag Name |
|---|---|
| 1 | N/A |
| ... | ... |
| 10000000000000000 | hideRings |
| 100000000000000000 | hideAmulet |
| ... | ... |
| 10000000000000000000000 | notPlayable |
| 100000000000000000000000 | heavyArmor |
Optional Subrecords / Fields are either implemented via MelNull (which will cause Wrye Bash to read the data, but discard it when writing) or via MelOptStruct (which will cause Wrye Bash to skip writing out struct entries that match the specified default value).
That's it for the 'crash course'. Much more detailed information is available on UESP and fopdoc, but be aware that those resources may be outdated or inaccurate - the only truly accurate sources are the xEdit Definitions. See the Further Reading section below for links to other pages on the wiki that offer further insight into this topic.
These are meant to be read in order: