UE4.27: Failed to parse ContainerHeader

[korgano]

I attempted to utilize the following commands on SD Gundam Battle Alliance, a UE 4.27.2 game, and received Failed to parse ContainerHeader errors:

.\retoc.exe manifest "E:\SteamLibrary\steamapps\common\SDGundamBA\SDGundamBA\Content\Paks\pakchunk0-WindowsNoEditor.utoc"

Failed to parse ContainerHeader. Package metadata will be unavailable: failed to fill whole buffer
thread 'main' panicked at src\main.rs:281:71:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

.\retoc.exe to-legacy --debug -v "E:\SteamLibrary\steamapps\common\SDGundamBA\SDGundamBA\Content\Paks\pakchunk0-WindowsNoEditor.utoc" -f GOP_* "E:\SteamLibrary\steamapps\common\SDGundamBA\SDGundamBA\Content\Paks\Extracted"

Failed to parse ContainerHeader. Package metadata will be unavailable: failed to fill whole bufferndam
[00:00:00] ########################################       0/0
Extracted 0 (0 failed) legacy assets to "E:\\SteamLibrary\\steamapps\\common\\SDGundamBA\\SDGundamBA\\Content\\Paks\\Extracted"
Extracted 0 shader code libraries to "E:\\SteamLibrary\\steamapps\\common\\SDGundamBA\\SDGundamBA\\Content\\Paks\\Extracted"

I am unsure why this occurs, as using retoc unpack-raw works on another of the game's UTOCs.

It may be related to the size of the UTOC, as pakchunk0-WindowsNoEditor.utoc is 2.2GB, and the UTOC dumped with unpack-raw was much smaller (sub-1GB).

[trumank]

Can you share the header? Can write it out to a file with this patch:

diff --git a/src/iostore.rs b/src/iostore.rs
index 16b9e9e..af6dfba 100644
--- a/src/iostore.rs
+++ b/src/iostore.rs
@@ -358,11 +358,12 @@ impl IoStoreContainer {
             .find(|info| info.id().get_chunk_type() == EIoChunkType::ContainerHeader);
         if let Some(header_chunk) = header_chunk {
             let data = container.read(header_chunk.id())?;
-            match FIoContainerHeader::de(&mut std::io::Cursor::new(data)) {
+            match FIoContainerHeader::de(&mut std::io::Cursor::new(&data)) {
                 Ok(header) => {
                     container.container_header = Some(header);
                 }
                 Err(err) => {
+                    fs::write("header.bin", &data)?;
                     eprintln!("Failed to parse ContainerHeader. Package metadata will be unavailable: {err:?}");
                 }
             }

[korgano]

I'll give it a shot and report back.

[korgano]

Here's the header.bin for the game.

SD_GBA_header.zip

[trumank]

So container header versions are kinda scuffed. Prior to UE5.0 the header did not have version info in it at all, which I thought wouldn't cause too much issue since I can rely on engine version/TOC version, but this game is a case in which that is not enough info. I tracked down this commit which landed at some point between 4.27.2 and 5.0.0 which removes ExportBundlesSize/Load Order/Pad from the container header. I will likely have to add a container header subversion and allow it to be manually specified by the user to handle this case.

[trumank]

Please try latest commit with --override-container-header-version PreInitial

[korgano]

Managed to extract the assets.

Would recommend updating the readme to explain that --override-container-header-version needs to be called between retoc and the function being used (to-legacy).

Also, are there any special instructions for the to-zen operation after editing assets? I've successfully packaged modded assets, but they cause the game to blackscreen/freeze on launch. Adding the -log option to the Steam launch options doesn't provide any output.

[trumank]

You would need to supply the same --override-container-header-version flag to to-zen, but that should it be it. It's possible there is still a bug with this container header version but I'd need the game to debug it.

[trumank]

I've verified modding assets works round trip:

retoc --override-container-header-version PreInitial to-legacy . unpacked
<edit unpacked/SDGundamBA/Content/Data/Gop/Text/en/GOP_Text_UMG_Static.uasset UAssetGUI>
retoc --override-container-header-version PreInitial to-zen unpacked mod_P.utoc --version UE4_27 --filter en/GOP_Text_UMG_Static

retoc --override-container-header-version PreInitial list mod_P.utoc --path
mod_P                          c01121ee256d25b100000002 ExportBundleData     ../../../SDGundamBA/Content/Data/Gop/Text/en/GOP_Text_UMG_Static.uasset
mod_P                          41e0df03eb6e30210000000a ContainerHeader      -

Menu screenshot

If you notice specific assets causing issues, please make a new issue for those.