Skip to content

Migration Guide

Jump to bottom Edit New page
John Stewart edited this page May 27, 2022 · 9 revisions

stability-beta

In most cases, migrating an existing CommonLibSSE project to use CommonLibSSE NG is fairly simple. This guide will cover using CommonLibSSE NG if you are currently using the standard upstream version, powerof3's fork, or CommonLibVR. This tutorial assumes your goal is to use CommonLibSSE NG's runtime multi-targeting and will address that topic. Other cases like unit testing features and Clang support are not covered here (yet).

Step 1: Building with Vcpkg

CommonLibSSE NG is intended to be consumed via Vcpkg, so migrating to Vcpkg for consuming CommonLibSSE can be your first step. The other upstream forks of CommonLibSSE mentioned above are all available through the Skyrim NG project's Vcpkg repository, making this fairly simple. First, add the Vcpkg repository to your project. To this create a vcpkg-configuration.json file in your project root (next to vcpkg.json). Add the following contents:

{
    "registries": [
        {
            "kind": "git",
            "repository": "https://gitlab.com/colorglass/vcpkg-colorglass",
            "baseline": "1a1a3c1ff3fc853cf7adf8c3475109763011d906",
            "packages": [
                "commonlibsse",
                "commonlibsse-po3-ae",
                "commonlibsse-po3-se",
                "commonlibvr",
                "commonlibsse-ng",
                "commonlibsse-ng-ae",
                "commonlibsse-ng-se",
                "commonlibsse-ng-vr",
                "commonlibsse-ng-flatrim"
            ]
        }
    ]
}

The baseline commit is the latest commit to the repository at https://gitlab.com/colorglass/vcpkg-colorglass. You should update it to whatever commit is latest at that repo (visit that page to check). Next, add a dependency on the correct CommonLibSSE in your vcpkg.json file to match the one you are currently using. The following options are supported:

  • commonlibsse: The original and standard CommonLibSSE, with exclusive support for AE (Skyrim SE 1.6.x).
  • commonlibsse-po3-ae: powerof3's fork, compiled to support AE (Skyrim SE 1.6.x).
  • commonlibsse-po3-se: powerof3's fork, compiled to support SE (Skyrim SE 1.5.x).
  • commonlibvr: Fork of CommonLibSSE for Skyrim VR.

For user's of powerof3's fork: it is recommended for now that you choose commonlibsse-po3-se. Having two separate builds for AE and SE has some detailed setup in CMake and Vcpkg when done through Vcpkg-based consumption, and for the sake of this migration we will ignore that since it becomes unnecessary in the final build. This means for now you will be building for SE only. If you have any build options to trigger the USING_AE definition, they can be removed, leaving the build SE-only.

Next, assuming you are consuming CommonLibSSE today as a Git submodule, then you should have the following line in your CMakeLists.txt:

add_subdirectory("path/to/CommonLibSSE")

This line should be removed and replaced with:

find_package(CommonLibSSE CONFIG REQUIRED)

Now reconfigure CMake and attempt to build. If all went well Vcpkg will now download and build CommonLibSSE for you, and your project will compile against it.

Step 2: Convert to CommonLibSSE NG

For the next step we will convert to CommonLibSSE NG, with only minimal to no multi-targeting of runtimes. If you use CommonLibVR currently, then this is simple. You will simply change your dependency in vcpkg.json from commonlibvr to commonlibsse-ng-vr. If you previously were using CommonLibSSE's upstream or powerof3 forks, then you should change this dependency to commonlibsse-ng-flatrim. Why "flatrim"? It's the meme name for non-VR Skyrim, and this build supports AE and SE, but not VR. We don't want to mix in VR to AE/SE (or AE/SE to VR) plugins yet, because there are some complications for backward compatibility. While 100% compatibility is not guaranteed between CommonLibSSE and CommonLibSSE NG Flatrim, nor CommonLibVR and CommonLibSSE NG VR, there are very few incompatible differences and most will never encounter them.

Step 3: Support AE and SE/VR Plugin Detection

In CommonLibSSE's upstream, as well as powerof3's fork built for AE, you can make your plugin detectable by SKSE for AE with a declaration such as the following:

EXTERN_C [[maybe_unused]] __declspec(dllexport) constinit auto SKSEPlugin_Version = []() noexcept {
    SKSE::PluginVersionData v;
    v.PluginName("MyPluginName");
    v.PluginVersion({ 1, 0, 0, 0 });
    v.UsesAddressLibrary(true);
    return v;
}();

In powerof3's CommonLibSSE for SE, and CommonLibVR, we instead use something like:

EXTERN_C [[maybe_unused]] __declspec(dllexport) bool SKSEAPI SKSEPlugin_Query(const QueryInterface*, PluginInfo* pluginInfo) {
    pluginInfo->name = "MyPluginName";
    pluginInfo->infoVersion = PluginInfo::kVersion;
    pluginInfo->version = REL::Version{1, 0, 0, 0}.pack();
    return true;
}

With CommonLibSSE NG, we can support all use cases, as the new data type used for AE is always available. Thus our plugin will expose both exports that all SKSE editions will search for, so any SKSE edition will always detect our plugin properly. Thus we can combine both options:

EXTERN_C [[maybe_unused]] __declspec(dllexport) constinit auto SKSEPlugin_Version = []() noexcept {
    SKSE::PluginVersionData v;
    v.PluginName("MyPluginName");
    v.PluginVersion({ 1, 0, 0, 0 });
    v.UsesAddressLibrary(true);
    return v;
}();

EXTERN_C [[maybe_unused]] __declspec(dllexport) bool SKSEAPI SKSEPlugin_Query(const QueryInterface*, PluginInfo* pluginInfo) {
    pluginInfo->name = SKSEPlugin_Version.pluginName;
    pluginInfo->infoVersion = PluginInfo::kVersion;
    pluginInfo->version = SKSEPlugin_Version.pluginVersion;
    return true;
}

Note that the SKSEPlugin_Query function here is kept minimal, only configuring the required plugin info and returning true. In the past it was common to do certain compatibility checks (e.g. to check if SKSE was loaded in the Creation Kit), or initialize logging, within this function. This is now moved to SKSEPlugin_Load, since AE will not have the SKSEPlugin_Query function execute. Thus we keep our SKSEPlugin_Query semantically similar to the effect of declaring SKSEPlugin_Version, and ensure the startup logic is effectively identical between SE/VR and AE.

CommonLibSSE NG includes some improvements that have been integrated from Fully Dynamic Game Engine's Trueflame library which can simplify the above declarations. First, while SKSE::PluginVersionData is kept for compatibility, the new SKSE::PluginDeclaration type can be used as its replacement. However, even this is not needed, since CommonLibSSE NG can generate and inject both the SKSEPlugin_Version and SKSEPlugin_Query functions through CMake. We do this by using add_commonlibsse_plugin in place of add_library in CMakeLists.txt.

find_package(CommonLibSSE CONFIG REQUIRED)

add_commonlibsse_plugin(${PROJECT_NAME}
    SOURCES ${headers} ${sources}
)

The above will create a shared library target, add CommonLibSSE as a link target (you do not need to specify it in target_link_libraries anymore), and inject a generated SKSEPlugin_Version and SKSEPlugin_Query. The plugin name defaults to the name of the target, and the version to the project version. Address Library is used for runtime compatibility mode by default. All of these values can be overridden. The full signature is:

add_commonlibsse_plugin(<target>
    # The plugin's name, defaults to target.
    NAME <string>

    # The plugin's author, empty by default.
    AUTHOR <string>

    # The support email address, empty by default.
    EMAIL <string>

    # The plugin version number, defaults to ${PROJECT_VERSION}.
    VERSION <version number>

    # Indicates the plugin is compatible with all runtimes via address library. This is the default if no
    # other compatibilility mode is specified. Can be used with USE_SIGNATURE_SCANNING but not
    # COMPATIBLE_RUNTIMES.
    USE_ADDRESS_LIBRARY

    # Indicates the plugin is compatible with all runtimes via signature scanning.  Can be used with
    # USE_ADDRESS_LIBRARY but not COMPATIBLE_RUNTIMES.
    USE_SIGNATURE_SCANNING

    # List of up to 16 Skyrim versions the plugin is compatible with. Cannot be used with
    # USE_ADDRESS_LIBRARY or USE_SIGNATURE_SCANNING.
    COMPATIBLE_RUNTIMES <version number> [<version number>...]

    # The minimum SKSE version to support; defaults to 0, and recommended by SKSE project to be left
    # 0.
    MINIMUM_SKSE_VERSION <version number>

    # Omit from all targets, same as used with add_library.
    EXCLUDE_FROM_ALL

    # List of the sources to include in the target, as would be the parameters to add_library.
    SOURCES <path> [<path>...]
)

Your SKSEPlugin_Load function definition can also be simplified for readability:

SKSEPluginLoad(LoadInterface* skse) {
    ...
    return true;
}

At this point we can now build our plugin. If we were previously targeting AE and/or SE, now we have a single DLL that can work on both. If were using CommonLibVR we still are VR only. In the next step we will attempt to bring together AE/SE and VR so all three can run in a single plugin.

Step 4: Full Runtime Multi-Targeting

Most reverse engineered content in CommonLibSSE works the same in VR and non-VR (at least as so far as is currently known), but there are some important exceptions. As long as you work with only VR or only non-VR runtimes, these exceptions will not affect you and you can use CommonLibSSE as you always have. But once you want to make a plugin work with VR and non-VR in a single plugin, things become more complex.

Some classes have a different memory layout in VR than non-VR Skyrim editions. They add a new member variable, which causes all subsequent variables in that class (and all subclasses) to have a different memory offset. It becomes impossible to compile a project that directly accesses these members, since at compile-time we don't know how to get the address of those members. Another problem is that several classes have new virtual functions, which has the same effect, but for the class' vtable. This changes the memory offsets of subsequent virtual functions (and newly-defined subclass virtual functions) to suffer the same problem, making them impossible to invoke. In the most extreme cases, Skyrim VR even has a number of new classes, and uses them as parent classes to other existing classes -- meaning some classes have different parent class hierarchies entirely in Skyrim VR than in Skyrim SE/AE. While the list of classes affected by this is relatively small, it includes many very important and commonly used ones:

  • RE::TESObjectREFR and all subclasses (e.g. RE::Actor, RE::Projectile, etc.).
  • RE::PlayerCharacter; while this is covered by being part of RE::TESObjectREFR this class is especially problematic because its memory layout in VR is only partially reverse engineered.
  • RE::IMenu and all subclasses, meaning you are likely affected if dealing with Skyrim UI.
  • RE::BSScript::IVirtualMachine and by extension RE::BSScript::Internal::VirtualMachine (note that basic native function registration is not affected, but if you interact with the Papyrus VM more intimately you likely are).
  • RE::NiAVObject and all subclasses, meaning if you work with meshes you are likely affected.
  • RE::BGSDefaultObjectManager.

Once you convert to having both VR and non-VR in the same project, you can no longer use these incompatible members or virtual functions in the same way. For the most part, virtual function invocation will be unaffected. CommonLibSSE NG exposes non-virtual functions as alternatives when using cross-VR runtime support. These non-virtual functions are used in a syntactically identical way, so they should work as a drop in replacement. The functions will dynamically lookup the proper function in the vtable based on the current runtime.

Member variable access is more complex. There is no way to support direct member variable access. Instead CommonLibSSE NG migrates blocks of member variables, which are identical in their contiguous memory layout between VR and non-VR, into structs. These structs are accessed by reference, returned form a call to an accessor function. These functions are often called GetRuntimeData(), and the struct they returned called RUNTIME_DATA. The memory address used in the return result is varied based on the current runtime, ensuring the struct provides access to the variables with the correct memory offset. In general all invalid direct member access should result in a compile-time error, so your task at this point is to compile, identify the errors, and look at each class to identify the accessor function to use. You will then need to change your code to use the indirect access. Note that in some complex cases there can be multiple such structs and multiple accessor functions, where there are multiple separate blocks of member variables identical between VR and non-VR (although this is not common).

Another use case, unique currently to the RE::IMenu class hierarchy, is upcasting complications. Due to the complex multiple inheritance in RE::IMenu types, and the different memory layouts, it is not safe to rely on static_cast (or implicit casting) to upcast to parent classes. In fact, in cross-VR builds of CommonLibSSE NG, any parent class that is not ABI-compatible in layout is not included in the class definition, making such upcasting impossible. As with member variable access, there are upcasting functions as an alternative, e.g. AsMenuEventHandler() to upcast to RE::MenuEventHandler*.

// Previously
void Foo(MenuEventHandler* handler, GFxValue root) { ... }
void Bar(FavoritesMenu* menu) {
    Foo(menu, menu->root);
}

// Now
void Foo(MenuEventHandler* handler, GFxValue root) { ... }
void Bar(FavoritesMenu* menu) {
    Foo(menu->AsMenuEventHandler(), menu->GetRuntimeData().root);
}

A major complication to adding VR support to existing AE/SE projects currently is the incomplete reverse engineering on RE::PlayerCharacter. Mods which depend heavily on this class should investigate if the functionality they depend on is part of the covered members. If not, new RE needs to be done to find that functionality or the mod will need to remain non-VR for the time being.

Add a custom footer
Add a custom sidebar
Clone this wiki locally