From d94cb20e44156f3d48a2ec9456e22aeae461ec03 Mon Sep 17 00:00:00 2001 From: Kurt Date: Thu, 29 Apr 2021 18:30:30 -0700 Subject: [PATCH] Initial commit --- .gitattributes | 2 + .gitignore | 251 +++++++++++++++++++++++++++ LICENSE | 15 ++ NewSnap.App/NewSnap.App.csproj | 13 ++ NewSnap.App/Program.cs | 89 ++++++++++ NewSnap.Lib/Compression/Oodle.cs | 37 ++++ NewSnap.Lib/DumpUtil.cs | 55 ++++++ NewSnap.Lib/Encryption/Adler32.cs | 40 +++++ NewSnap.Lib/Encryption/Crc32.cs | 70 ++++++++ NewSnap.Lib/Encryption/XorShift.cs | 123 +++++++++++++ NewSnap.Lib/MSBT/BinaryReaderX.cs | 53 ++++++ NewSnap.Lib/MSBT/ByteOrder.cs | 8 + NewSnap.Lib/MSBT/LBL1.cs | 17 ++ NewSnap.Lib/MSBT/MSBT.cs | 156 +++++++++++++++++ NewSnap.Lib/MSBT/MSBTEncodingByte.cs | 8 + NewSnap.Lib/MSBT/MSBTGroup.cs | 8 + NewSnap.Lib/MSBT/MSBTHeader.cs | 46 +++++ NewSnap.Lib/MSBT/MSBTLabel.cs | 21 +++ NewSnap.Lib/MSBT/MSBTSection.cs | 15 ++ NewSnap.Lib/MSBT/MSBTTextString.cs | 48 +++++ NewSnap.Lib/MSBT/MSBTUtil.cs | 67 +++++++ NewSnap.Lib/MSBT/TXT2.cs | 16 ++ NewSnap.Lib/Models/DrpArchive.cs | 244 ++++++++++++++++++++++++++ NewSnap.Lib/Models/DrpFileEntry.cs | 137 +++++++++++++++ NewSnap.Lib/NewSnap.Lib.csproj | 9 + NewSnap.Lib/Saves/SaveDumper.cs | 79 +++++++++ NewSnap.Lib/Saves/SaveFile.cs | 75 ++++++++ NewSnap.Lib/Saves/SaveFileEntry.cs | 33 ++++ NewSnap.Lib/Saves/SaveFileHeader.cs | 29 ++++ NewSnap.Lib/Saves/SaveReader.cs | 136 +++++++++++++++ NewSnap.Tests/CrcTests.cs | 25 +++ NewSnap.Tests/MSBTTests.cs | 41 +++++ NewSnap.Tests/NewSnap.Tests.csproj | 25 +++ NewSnap.Tests/SaveTests.cs | 124 +++++++++++++ NewSnap.sln | 37 ++++ README.md | 19 ++ 36 files changed, 2171 insertions(+) create mode 100644 .gitattributes create mode 100644 .gitignore create mode 100644 LICENSE create mode 100644 NewSnap.App/NewSnap.App.csproj create mode 100644 NewSnap.App/Program.cs create mode 100644 NewSnap.Lib/Compression/Oodle.cs create mode 100644 NewSnap.Lib/DumpUtil.cs create mode 100644 NewSnap.Lib/Encryption/Adler32.cs create mode 100644 NewSnap.Lib/Encryption/Crc32.cs create mode 100644 NewSnap.Lib/Encryption/XorShift.cs create mode 100644 NewSnap.Lib/MSBT/BinaryReaderX.cs create mode 100644 NewSnap.Lib/MSBT/ByteOrder.cs create mode 100644 NewSnap.Lib/MSBT/LBL1.cs create mode 100644 NewSnap.Lib/MSBT/MSBT.cs create mode 100644 NewSnap.Lib/MSBT/MSBTEncodingByte.cs create mode 100644 NewSnap.Lib/MSBT/MSBTGroup.cs create mode 100644 NewSnap.Lib/MSBT/MSBTHeader.cs create mode 100644 NewSnap.Lib/MSBT/MSBTLabel.cs create mode 100644 NewSnap.Lib/MSBT/MSBTSection.cs create mode 100644 NewSnap.Lib/MSBT/MSBTTextString.cs create mode 100644 NewSnap.Lib/MSBT/MSBTUtil.cs create mode 100644 NewSnap.Lib/MSBT/TXT2.cs create mode 100644 NewSnap.Lib/Models/DrpArchive.cs create mode 100644 NewSnap.Lib/Models/DrpFileEntry.cs create mode 100644 NewSnap.Lib/NewSnap.Lib.csproj create mode 100644 NewSnap.Lib/Saves/SaveDumper.cs create mode 100644 NewSnap.Lib/Saves/SaveFile.cs create mode 100644 NewSnap.Lib/Saves/SaveFileEntry.cs create mode 100644 NewSnap.Lib/Saves/SaveFileHeader.cs create mode 100644 NewSnap.Lib/Saves/SaveReader.cs create mode 100644 NewSnap.Tests/CrcTests.cs create mode 100644 NewSnap.Tests/MSBTTests.cs create mode 100644 NewSnap.Tests/NewSnap.Tests.csproj create mode 100644 NewSnap.Tests/SaveTests.cs create mode 100644 NewSnap.sln create mode 100644 README.md diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 0000000..dfe0770 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,2 @@ +# Auto detect text files and perform LF normalization +* text=auto diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..b00ec75 --- /dev/null +++ b/.gitignore @@ -0,0 +1,251 @@ +################# +## Eclipse +################# + +*.pydevproject +.project +.metadata +bin/ +tmp/ +*.tmp +*.bak +*.swp +*~.nib +local.properties +.classpath +.settings/ +.loadpath + +# External tool builders +.externalToolBuilders/ + +# Locally stored "Eclipse launch configurations" +*.launch + +# CDT-specific +.cproject + +# PDT-specific +.buildpath + + +################# +## Visual Studio +################# + +## Ignore Visual Studio temporary files, build results, and +## files generated by popular Visual Studio add-ons. + +# User-specific files +*.suo +*.user +*.sln.docstates +*.vs + +# Build results + +[Dd]ebug/ +[Rr]elease/ +x64/ +build/ +[Bb]in/ +[Oo]bj/ + +# MSTest test Results +[Tt]est[Rr]esult*/ +[Bb]uild[Ll]og.* + +*_i.c +*_p.c +*.ilk +*.meta +*.obj +*.pch +*.pdb +*.pgc +*.pgd +*.rsp +*.sbr +*.tlb +*.tli +*.tlh +*.tmp +*.tmp_proj +*.log +*.vspscc +*.vssscc +.builds +*.pidb +*.log +*.scc + +# Visual C++ cache files +ipch/ +*.aps +*.ncb +*.opensdf +*.sdf +*.cachefile + +# Visual Studio profiler +*.psess +*.vsp +*.vspx + +# Guidance Automation Toolkit +*.gpState + +# ReSharper is a .NET coding add-in +_ReSharper*/ +*.[Rr]e[Ss]harper + +# TeamCity is a build add-in +_TeamCity* + +# DotCover is a Code Coverage Tool +*.dotCover + +# NCrunch +*.ncrunch* +.*crunch*.local.xml + +# Installshield output folder +[Ee]xpress/ + +# DocProject is a documentation generator add-in +DocProject/buildhelp/ +DocProject/Help/*.HxT +DocProject/Help/*.HxC +DocProject/Help/*.hhc +DocProject/Help/*.hhk +DocProject/Help/*.hhp +DocProject/Help/Html2 +DocProject/Help/html + +# Click-Once directory +publish/ + +# Publish Web Output +*.Publish.xml +*.pubxml + +# NuGet Packages Directory +## TODO: If you have NuGet Package Restore enabled, uncomment the next line +packages/ + +# Windows Azure Build Output +csx +*.build.csdef + +# Windows Store app package directory +AppPackages/ + +# Others +sql/ +*.Cache +ClientBin/ +[Ss]tyle[Cc]op.* +~$* +*~ +*.dbmdl +*.[Pp]ublish.xml +*.pfx +*.publishsettings + +# RIA/Silverlight projects +Generated_Code/ + +# Backup & report files from converting an old project file to a newer +# Visual Studio version. Backup files are not needed, because we have git ;-) +_UpgradeReport_Files/ +Backup*/ +UpgradeLog*.XML +UpgradeLog*.htm + +# SQL Server files +App_Data/*.mdf +App_Data/*.ldf + + +############# +## Qt Creator +############# + +*.save +*.autosave + +############# +## GNU Emacs +############# + +\#* +.\#* +*.elc + + +############# +## Windows detritus +############# + +# Windows image file caches +Thumbs.db +ehthumbs.db + +# Folder config file +Desktop.ini + +# Recycle Bin used on file shares +$RECYCLE.BIN/ + + +############# +## OS X +############# + +.DS_Store + + +############# +## C# +############# + +*.resources +pingme.txt + + +############# +## Python +############# + +*.py[co] + +# Packages +*.egg +*.egg-info +dist/ +build/ +eggs/ +parts/ +var/ +sdist/ +develop-eggs/ +.installed.cfg + +# Installer logs +pip-log.txt + +# Unit test / coverage reports +.coverage +.tox + +# Translations +*.mo + +# Mr Developer +.mr.developer.cfg + +################# +## MonoDevelop +################# + +*.userprefs diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..b72712b --- /dev/null +++ b/LICENSE @@ -0,0 +1,15 @@ +ISC License + +Copyright (c) 2021, SciresM, Kaphotics + +Permission to use, copy, modify, and/or distribute this software for any +purpose with or without fee is hereby granted, provided that the above +copyright notice and this permission notice appear in all copies. + +THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. diff --git a/NewSnap.App/NewSnap.App.csproj b/NewSnap.App/NewSnap.App.csproj new file mode 100644 index 0000000..c662283 --- /dev/null +++ b/NewSnap.App/NewSnap.App.csproj @@ -0,0 +1,13 @@ + + + + Exe + netcoreapp3.1 + 9 + + + + + + + diff --git a/NewSnap.App/Program.cs b/NewSnap.App/Program.cs new file mode 100644 index 0000000..aad08f3 --- /dev/null +++ b/NewSnap.App/Program.cs @@ -0,0 +1,89 @@ +using System; +using System.IO; +using NewSnap.Lib; + +namespace NewSnap.App +{ + internal static class Program + { + private static void Main(string[] args) + { + if (args.Length is not (2 or 3)) + { + PrintUsage(); + return; + } + + try + { + Dump(args); + } +#pragma warning disable CA1031 // Do not catch general exception types + catch (Exception ex) +#pragma warning restore CA1031 // Do not catch general exception types + { + Console.WriteLine(ex); + } + } + + private static void Dump(string[] args) + { + var mode = args[0]; + var path = args[1]; + switch (mode) + { + case "-sav" when !File.Exists(path): + Console.WriteLine("Input ROM directory not found."); + return; + + case "-sav": + { + var dest = args.Length == 3 ? args[2] : path; + SaveDumper.ExtractEntries(path, dest); + break; + } + + case "-drp" when Directory.Exists(path): + { + var dest = args.Length == 3 ? args[2] : path; + DumpUtil.DumpAllDrp(path, dest); + break; + } + + case "-drp" when !File.Exists(path): + Console.WriteLine("Input drp file not found."); + return; + + case "-drp": + { + var dest = args.Length == 3 ? args[2] : Path.GetFullPath(path); + DumpUtil.DumpToPath(path, dest); + break; + } + + default: + PrintUsage(); + return; + } + + Console.WriteLine("Done!"); + } + + private static void PrintUsage() + { + Console.WriteLine(@$"{nameof(NewSnap)} Command Line +============================== + +See below for command line parameters. +An optional destination path will resolve to the source file/folder's current folder if not provided. + +============================== +-sav [folder] [destFolder(Optional)] +-drp [drpFile] [destFolder(Optional)] +-drp [drpFolder] [destFolder(Optional)] +============================== +Hint: [x] are string paths. +"); + } + } +} diff --git a/NewSnap.Lib/Compression/Oodle.cs b/NewSnap.Lib/Compression/Oodle.cs new file mode 100644 index 0000000..659ee8b --- /dev/null +++ b/NewSnap.Lib/Compression/Oodle.cs @@ -0,0 +1,37 @@ +using System.Runtime.InteropServices; + +namespace NewSnap.Lib +{ + public static class Oodle + { + /// + /// Oodle Library Path + /// + private const string OodleLibraryPath = "oo2core_8_win64"; + + /// + /// Oodle64 Decompression Method + /// + [DllImport(OodleLibraryPath, CallingConvention = CallingConvention.Cdecl)] + private static extern long OodleLZ_Decompress(byte[] buffer, long bufferSize, byte[] result, long outputBufferSize, int a, int b, int c, long d, long e, long f, long g, long h, long i, int ThreadModule); + + /// + /// Decompresses a byte array of Oodle Compressed Data (Requires Oodle DLL) + /// + /// Input Compressed Data + /// Decompressed Size + /// Resulting Array if success, otherwise null. + public static byte[]? Decompress(byte[] input, long decompressedLength) + { + // Resulting decompressed Data + byte[] result = new byte[decompressedLength]; + // Decode the data (other parameters such as callbacks not required) + long decodedSize = OodleLZ_Decompress(input, input.Length, result, decompressedLength, 1, 0, 0, 0, 0, 0, 0, 0, 0, 3); + // Check did we fail + if (decodedSize == 0) + return null; + // Return Result + return result; + } + } +} diff --git a/NewSnap.Lib/DumpUtil.cs b/NewSnap.Lib/DumpUtil.cs new file mode 100644 index 0000000..de4c9f3 --- /dev/null +++ b/NewSnap.Lib/DumpUtil.cs @@ -0,0 +1,55 @@ +using System; +using System.IO; + +namespace NewSnap.Lib +{ + public static class DumpUtil + { + private static void EnsureEndsWithDirectorySeparator(ref string path) + { + path = Path.Combine(path, "x"); + path = path[..^1]; + } + + public static void DumpAllDrp(string path, string outDir) + { + EnsureEndsWithDirectorySeparator(ref path); + EnsureEndsWithDirectorySeparator(ref outDir); + string[] files = Directory.GetFiles(path, "*.drp", SearchOption.AllDirectories); + for (var i = 0; i < files.Length; i++) + { + var f = files[i]; + var fi = new FileInfo(f); + var src = fi.Directory.FullName; + var dest = src.Replace(path, outDir); + + Console.WriteLine($"{i + 1}/{files.Length}: Reading and decrypting {f}"); + DumpToPath(f, dest); + } + } + + public static void DumpToPath(string path, string outDir) + { + var arcData = File.ReadAllBytes(path); + var archive = new DrpArchive(arcData); + Console.WriteLine($"Dumping {archive.FileCount} files from {path}"); + var fn = Path.GetFileNameWithoutExtension(path); + var subDir = Path.Combine(outDir, fn); + DumpToPath(archive, subDir); + } + + private static void DumpToPath(DrpArchive archive, string outDir) + { + Directory.CreateDirectory(outDir); + for (var i = 0; i < archive.FileCount; ++i) + { + var name = archive.GetFileName(i); + var dest = Path.Combine(outDir, name); + var data = archive.GetFileData(i); + File.WriteAllBytes(dest, data); + + Console.WriteLine($"\t{i + 1}/{archive.FileCount}: {dest}"); + } + } + } +} diff --git a/NewSnap.Lib/Encryption/Adler32.cs b/NewSnap.Lib/Encryption/Adler32.cs new file mode 100644 index 0000000..473f10c --- /dev/null +++ b/NewSnap.Lib/Encryption/Adler32.cs @@ -0,0 +1,40 @@ +using System; + +namespace NewSnap.Lib +{ + public static class Adler32 + { + private const uint MOD_ADLER = 65521; + + public static uint ComputeChecksum(ReadOnlySpan arr, int offset, int count) + { + var a = 1u; + var b = 0u; + + for (var i = 0; i < count; ++i) + { + a = (a + arr[offset + i]) % MOD_ADLER; + b = (b + a) % MOD_ADLER; + } + + return (b << 16) | a; + } + + public static uint ComputeChecksum(string arr, int offset, int count) + { + var a = 1u; + var b = 0u; + + for (var i = 0; i < count; ++i) + { + a = (a + (byte)arr[offset + i]) % MOD_ADLER; + b = (b + a) % MOD_ADLER; + } + + return (b << 16) | a; + } + + public static uint ComputeChecksum(ReadOnlySpan arr) => ComputeChecksum(arr, 0, arr.Length); + public static uint ComputeChecksum(string arr) => ComputeChecksum(arr, 0, arr.Length); + } +} diff --git a/NewSnap.Lib/Encryption/Crc32.cs b/NewSnap.Lib/Encryption/Crc32.cs new file mode 100644 index 0000000..16b3a01 --- /dev/null +++ b/NewSnap.Lib/Encryption/Crc32.cs @@ -0,0 +1,70 @@ +using System; + +namespace NewSnap.Lib +{ + public static class Crc32 + { + private static readonly uint[] CrcTable = + { + 0x00000000, 0x77073096, 0xee0e612c, 0x990951ba, 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3, + 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988, 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91, + 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de, 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7, + 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec, 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5, + 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172, 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b, + 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940, 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59, + 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116, 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f, + 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924, 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d, + 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a, 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433, + 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818, 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01, + 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e, 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457, + 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c, 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65, + 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2, 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb, + 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0, 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9, + 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086, 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f, + 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4, 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad, + 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a, 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683, + 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8, 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1, + 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe, 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7, + 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc, 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5, + 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252, 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b, + 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60, 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79, + 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236, 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f, + 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04, 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d, + 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a, 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713, + 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38, 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21, + 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e, 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777, + 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c, 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45, + 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2, 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db, + 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0, 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9, + 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6, 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf, + 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94, 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d + }; + + public static uint ComputeChecksum(ReadOnlySpan arr, int offset, int count) + { + var table = CrcTable; + var checksum = ~0u; + for (var i = 0; i < count; ++i) + { + checksum = (checksum >> 8) ^ table[(checksum & 0xFF) ^ arr[offset + i]]; + } + + return ~checksum; + } + + public static uint ComputeChecksum(string arr, int offset, int count) + { + var table = CrcTable; + var checksum = ~0u; + for (var i = 0; i < count; ++i) + { + checksum = (checksum >> 8) ^ table[(checksum & 0xFF) ^ (byte)arr[offset + i]]; + } + + return ~checksum; + } + + public static uint ComputeChecksum(ReadOnlySpan arr) => ComputeChecksum(arr, 0, arr.Length); + public static uint ComputeChecksum(string arr) => ComputeChecksum(arr, 0, arr.Length); + } +} diff --git a/NewSnap.Lib/Encryption/XorShift.cs b/NewSnap.Lib/Encryption/XorShift.cs new file mode 100644 index 0000000..99de2e6 --- /dev/null +++ b/NewSnap.Lib/Encryption/XorShift.cs @@ -0,0 +1,123 @@ +using System.Diagnostics; + +namespace NewSnap.Lib +{ + /// + /// 128bit XorShift RNG, seeded from a 32bit value expanded into 128bits. + /// + public class XorShift + { + private const uint Mult = 0x41C64E6D; + private const uint Add = 12345; + private uint _x; + private uint _y; + private uint _z; + private uint _w; + + public XorShift(uint seed) + { + _w = (Mult * seed) + Add; + _z = (Mult * _w) + Add; + _y = (Mult * _z) + Add; + _x = (Mult * _y) + Add; + } + + public XorShift(ulong lo, ulong hi) + { + _w = (uint)lo; + _z = (uint)(lo >> 32); + _y = (uint)hi; + _x = (uint)(hi >> 32); + } + + public uint GetNext() + { + var t = _x ^ (_x << 11); + _x = _y; + _y = _z; + _z = _w; + _w = _w ^ (uint)((int)_w >> 19) ^ (t ^ (uint)((int)t >> 8)); + return _w & 0x7FFFFFFF; + } + + /// + /// Gets the next random number. + /// + /// Inclusive maximum? + /// + public uint GetNext(uint limit) => limit switch + { + 0 => 0, + 0x7FFFFFFF => GetNext(), + >= 0x80000000 => GetNextUnsigned(limit), + _ => GetNextWithin(limit) + }; + + private uint GetNextWithin(uint limit) + { + Debug.Assert(limit < int.MaxValue); + var divisor = 0x80000000 / (limit + 1); + while (true) + { + var result = GetNext() / divisor; + if (result <= limit) + return result; + } + } + + private uint GetNextUnsigned(uint limit) + { + Debug.Assert(limit > int.MaxValue); + // This method is inlined and can't be called without a high enough limit. + // Some of the disassembly optimizations / modifications depend on this. + + const uint MaxLimit = 0xFFFFFFFF; + const int MaxLimitSigned = int.MaxValue; + + // There's a little bit of optimization to get the result via multiple calls. + var maximum = limit + 1; + var passes = (limit == MaxLimit) ? 2u : 1u; // if max, 2, otherwise (x+1) >> 31, which is always 1 for our inlined logic. + var result = 0u; + var carry = 1u; + while (true) + { + result += GetNext() * carry; + + // Edge case for limit = 0x80000000 and 0x80000001 + // 0x7FFFFFFF * 0x00000001 == 0x80000000 - 1 (first iteration) + // 0x7FFFFFFF * 0x80000000 == 0x80000001 - 1 (second iteration) + if (MaxLimitSigned * carry == maximum - carry) + return result; + + // This check is kinda useless with the inlined logic, since it is never 0. + carry <<= 31; + if (carry <= passes) + continue; + + var next = GetNext(limit / carry); + if (next <= MaxLimit / carry) + { + var res = result + (next * carry); + if (res >= result && res <= limit) + return res; + } + + // Went over our limit! Try again. + result = 0; + carry = 1; + } + } + } + + public static class XorShiftUtil + { + public static void DecryptWord(this XorShift rng, byte[] archive, int offset) + { + uint rand = rng.GetNext(0xFFFFFFFF); + archive[offset + 0] ^= (byte)(rand >> 00); + archive[offset + 1] ^= (byte)(rand >> 08); + archive[offset + 2] ^= (byte)(rand >> 16); + archive[offset + 3] ^= (byte)(rand >> 24); + } + } +} diff --git a/NewSnap.Lib/MSBT/BinaryReaderX.cs b/NewSnap.Lib/MSBT/BinaryReaderX.cs new file mode 100644 index 0000000..3c03756 --- /dev/null +++ b/NewSnap.Lib/MSBT/BinaryReaderX.cs @@ -0,0 +1,53 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Text; + +namespace Lib +{ + internal class BinaryReaderX : BinaryReader + { + public ByteOrder ByteOrder { get; set; } + + public BinaryReaderX(Stream input, ByteOrder byteOrder = ByteOrder.LittleEndian) + : base(input) + { + ByteOrder = byteOrder; + } + + public override ushort ReadUInt16() + { + if (ByteOrder == ByteOrder.LittleEndian) + return base.ReadUInt16(); + else + return BitConverter.ToUInt16(base.ReadBytes(2).Reverse().ToArray(), 0); + } + + public override uint ReadUInt32() + { + if (ByteOrder == ByteOrder.LittleEndian) + return base.ReadUInt32(); + else + return BitConverter.ToUInt32(base.ReadBytes(4).Reverse().ToArray(), 0); + } + + public string ReadString(int length) + { + return Encoding.ASCII.GetString(ReadBytes(length)).TrimEnd('\0'); + } + + public string PeekString(int length = 4) + { + List bytes = new(); + long startOffset = BaseStream.Position; + + for (int i = 0; i < length; i++) + bytes.Add(ReadByte()); + + BaseStream.Seek(startOffset, SeekOrigin.Begin); + + return Encoding.ASCII.GetString(bytes.ToArray()); + } + } +} diff --git a/NewSnap.Lib/MSBT/ByteOrder.cs b/NewSnap.Lib/MSBT/ByteOrder.cs new file mode 100644 index 0000000..d45b96b --- /dev/null +++ b/NewSnap.Lib/MSBT/ByteOrder.cs @@ -0,0 +1,8 @@ +namespace Lib +{ + public enum ByteOrder + { + LittleEndian = 0, + BigEndian = 1 + } +} diff --git a/NewSnap.Lib/MSBT/LBL1.cs b/NewSnap.Lib/MSBT/LBL1.cs new file mode 100644 index 0000000..10b596f --- /dev/null +++ b/NewSnap.Lib/MSBT/LBL1.cs @@ -0,0 +1,17 @@ +using System; +using System.Collections.Generic; + +namespace Lib +{ + public class LBL1 : MSBTSection + { + public uint NumberOfGroups; + + public readonly List Groups = new(); + public readonly List Labels = new(); + + public LBL1() : base(string.Empty, Array.Empty()) + { + } + } +} diff --git a/NewSnap.Lib/MSBT/MSBT.cs b/NewSnap.Lib/MSBT/MSBT.cs new file mode 100644 index 0000000..49757a0 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBT.cs @@ -0,0 +1,156 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Text; + +namespace Lib +{ + public class MSBT + { + public readonly MSBTHeader Header; + public readonly LBL1 LBL1 = new(); + public readonly TXT2 TXT2 = new(); + public readonly Encoding FileEncoding; + public readonly List SectionOrder; + public bool HasLabels; + + public MSBT(byte[] rawBytes) + { + using var stream = new MemoryStream(rawBytes); + using var br = new BinaryReaderX(stream); + + Header = new MSBTHeader(br); + FileEncoding = (Header.EncodingByte == MSBTEncodingByte.UTF8 ? Encoding.UTF8 : Encoding.Unicode); + + SectionOrder = new List(); + for (int i = 0; i < Header.NumberOfSections; i++) + { + var peek = br.PeekString(); + switch (peek) + { + case "LBL1": + ReadLBL1(br); + SectionOrder.Add("LBL1"); + break; + + case "ATR1": + // don't care + var magic = br.ReadUInt32(); + var size = br.ReadUInt32(); + var Padding = br.ReadBytes(8); + var Data = br.ReadBytes((int)size); + while (br.BaseStream.Position % 16 != 0 && br.BaseStream.Position != br.BaseStream.Length) + br.ReadByte(); + SectionOrder.Add("ATR1"); + break; + + case "TXT2": + ReadTXT2(br); + SectionOrder.Add("TXT2"); + break; + + default: + i--; + br.ReadUInt32(); + break; + } + } + + br.Close(); + } + + private void ReadLBL1(BinaryReaderX br) + { + LBL1.Identifier = br.ReadString(4); + LBL1.SectionSize = br.ReadUInt32(); + LBL1.Padding1 = br.ReadBytes(8); + long startOfLabels = br.BaseStream.Position; + LBL1.NumberOfGroups = br.ReadUInt32(); + + for (int i = 0; i < LBL1.NumberOfGroups; i++) + { + var grp = new MSBTGroup + { + NumberOfLabels = br.ReadUInt32(), + Offset = br.ReadUInt32() + }; + LBL1.Groups.Add(grp); + } + + foreach (var grp in LBL1.Groups) + { + br.BaseStream.Seek(startOfLabels + grp.Offset, SeekOrigin.Begin); + + for (int i = 0; i < grp.NumberOfLabels; i++) + { + var length = Convert.ToUInt32(br.ReadByte()); + var name = br.ReadString((int)length); + var index = br.ReadUInt32(); + var lbl = new MSBTLabel(name) {Index = index, Length = length}; + LBL1.Labels.Add(lbl); + } + } + + if (LBL1.Labels.Count > 0) + HasLabels = true; + + PaddingSeek(br); + } + + private void ReadTXT2(BinaryReaderX br) + { + TXT2.Identifier = br.ReadString(4); + TXT2.SectionSize = br.ReadUInt32(); + TXT2.Padding1 = br.ReadBytes(8); + long startOfStrings = br.BaseStream.Position; + TXT2.NumberOfStrings = br.ReadUInt32(); + + var offsets = new List(); + for (int i = 0; i < TXT2.NumberOfStrings; i++) + offsets.Add(br.ReadUInt32()); + + for (int i = 0; i < TXT2.NumberOfStrings; i++) + { + uint nextOffset = (i + 1 < offsets.Count) ? ((uint)startOfStrings + offsets[i + 1]) : ((uint)startOfStrings + TXT2.SectionSize); + + br.BaseStream.Seek(startOfStrings + offsets[i], SeekOrigin.Begin); + + var result = new List(); + while (br.BaseStream.Position < nextOffset && br.BaseStream.Position < Header.FileSize) + { + if (Header.EncodingByte == MSBTEncodingByte.UTF8) + { + result.Add(br.ReadByte()); + } + else + { + byte[] unichar = br.ReadBytes(2); + + if (br.ByteOrder == ByteOrder.BigEndian) + Array.Reverse(unichar); + + result.AddRange(unichar); + } + } + var str = new MSBTTextString(result.ToArray(), (uint)i); + TXT2.Strings.Add(str); + } + + // Tie in LBL1 labels + foreach (var lbl in LBL1.Labels) + lbl.String = TXT2.Strings[(int)lbl.Index]; + + PaddingSeek(br); + } + + private static void PaddingSeek(BinaryReader br) + { + long remainder = br.BaseStream.Position % 16; + if (remainder <= 0) + return; + var _ = br.ReadByte(); + br.BaseStream.Seek(-1, SeekOrigin.Current); + br.BaseStream.Seek(16 - remainder, SeekOrigin.Current); + } + } +} diff --git a/NewSnap.Lib/MSBT/MSBTEncodingByte.cs b/NewSnap.Lib/MSBT/MSBTEncodingByte.cs new file mode 100644 index 0000000..f291c34 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTEncodingByte.cs @@ -0,0 +1,8 @@ +namespace Lib +{ + public enum MSBTEncodingByte : byte + { + UTF8 = 0x00, + Unicode = 0x01 + } +} diff --git a/NewSnap.Lib/MSBT/MSBTGroup.cs b/NewSnap.Lib/MSBT/MSBTGroup.cs new file mode 100644 index 0000000..10ce1e2 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTGroup.cs @@ -0,0 +1,8 @@ +namespace Lib +{ + public class MSBTGroup + { + public uint NumberOfLabels; + public uint Offset; + } +} diff --git a/NewSnap.Lib/MSBT/MSBTHeader.cs b/NewSnap.Lib/MSBT/MSBTHeader.cs new file mode 100644 index 0000000..87381f6 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTHeader.cs @@ -0,0 +1,46 @@ +using System; + +namespace Lib +{ + public class MSBTHeader + { + public readonly string Identifier; // MsgStdBn + public readonly byte[] ByteOrderMark; + public readonly ushort Unknown1; // Always 0x0000 + public readonly MSBTEncodingByte EncodingByte; + public readonly byte Unknown2; // Always 0x03 + public readonly ushort NumberOfSections; + public readonly ushort Unknown3; // Always 0x0000 + public readonly uint FileSize; + public readonly byte[] Unknown4; // Always 0x0000 0000 0000 0000 0000 + + public uint FileSizeOffset; + + internal MSBTHeader(BinaryReaderX br) + { + // Header + Identifier = br.ReadString(8); + if (Identifier != "MsgStdBn") + throw new ArgumentException("The file provided is not a valid MSBT file."); + + // Byte Order + ByteOrderMark = br.ReadBytes(2); + br.ByteOrder = ByteOrderMark[0] > ByteOrderMark[1] ? ByteOrder.LittleEndian : ByteOrder.BigEndian; + + Unknown1 = br.ReadUInt16(); + + // Encoding + EncodingByte = (MSBTEncodingByte)br.ReadByte(); + + Unknown2 = br.ReadByte(); + NumberOfSections = br.ReadUInt16(); + Unknown3 = br.ReadUInt16(); + FileSizeOffset = (uint)br.BaseStream.Position; // Record offset for future use + FileSize = br.ReadUInt32(); + Unknown4 = br.ReadBytes(10); + + if (FileSize != br.BaseStream.Length) + throw new ArgumentException("The file provided is not a valid MSBT file."); + } + } +} diff --git a/NewSnap.Lib/MSBT/MSBTLabel.cs b/NewSnap.Lib/MSBT/MSBTLabel.cs new file mode 100644 index 0000000..8095cd0 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTLabel.cs @@ -0,0 +1,21 @@ +using System.Text; + +namespace Lib +{ + public class MSBTLabel + { + public uint Length; + public readonly string Name; + public MSBTTextString String; + + public MSBTLabel(string name) + { + Name = name; + String = MSBTTextString.Empty; + } + + public uint Index { get; set; } + public override string ToString() => Length > 0 ? Name : (Index + 1).ToString(); + public string ToString(Encoding encoding) => encoding.GetString(String.Value); + } +} diff --git a/NewSnap.Lib/MSBT/MSBTSection.cs b/NewSnap.Lib/MSBT/MSBTSection.cs new file mode 100644 index 0000000..194a247 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTSection.cs @@ -0,0 +1,15 @@ +namespace Lib +{ + public class MSBTSection + { + public string Identifier; + public uint SectionSize; // Begins after Unknown1 + public byte[] Padding1; // Always 0x0000 0000 + + public MSBTSection(string identifier, byte[] padding) + { + Identifier = identifier; + Padding1 = padding; + } + } +} diff --git a/NewSnap.Lib/MSBT/MSBTTextString.cs b/NewSnap.Lib/MSBT/MSBTTextString.cs new file mode 100644 index 0000000..cc5c454 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTTextString.cs @@ -0,0 +1,48 @@ +using System; +using System.Text; + +namespace Lib +{ + public class MSBTTextString + { + public readonly byte[] Value; + public readonly uint Index; + + public static readonly MSBTTextString Empty = new(Array.Empty(), 0); + + public MSBTTextString(byte[] v, uint i) + { + Value = v; + Index = i; + } + + public override string ToString() => (Index + 1).ToString(); + + public string ToString(Encoding encoding) => encoding.GetString(Value); + + public string ToStringNoAtoms() => GetTextWithoutAtoms(Value); + + public static string GetTextWithoutAtoms(byte[] data) + { + var sb = new StringBuilder(); + for (int i = 0; i < data.Length; i += 2) + { + char c = BitConverter.ToChar(data, i); + if (c == 0xE) // atom + { + // skip over atom and the u16,u16 + i += 2 * 3; + var len = BitConverter.ToUInt16(data, i); + i += len; // skip over extra atom data + continue; + } + + if (c == '\0') + break; + sb.Append(c); + } + + return sb.ToString(); + } + } +} diff --git a/NewSnap.Lib/MSBT/MSBTUtil.cs b/NewSnap.Lib/MSBT/MSBTUtil.cs new file mode 100644 index 0000000..cb79882 --- /dev/null +++ b/NewSnap.Lib/MSBT/MSBTUtil.cs @@ -0,0 +1,67 @@ +using System.Collections.Generic; +using System.Diagnostics; +using System.IO; +using System.Linq; + +namespace Lib +{ + public static class MSBTUtil + { + public static void DebugDumpLines(this MSBT obj) + { + var lines = obj.GetOrderedLines(); + foreach (var line in lines) + Debug.WriteLine(line); + } + + public static IEnumerable GetOrderedLines(string path, int indexBias = 0) => new MSBT(File.ReadAllBytes(path)).GetOrderedLines(indexBias); + + public static IEnumerable GetOrderedLines(this MSBT obj, int indexBias = 0) + { + var sorted = obj.LBL1.Labels + .Where(z => !z.Name.EndsWith("_pl")) + .OrderBy(z => z.Index); + + foreach (var x in sorted) + { + var index = x.Index; + var name = x.Name; + var data = obj.TXT2.Strings[(int)index]; + var line = data.ToString(obj.FileEncoding).TrimEnd('\0'); + yield return $"{line} = {index + indexBias}, // {name}"; + } + } + + public static IEnumerable GetOrderedLinesTab(this MSBT obj) + { + var sorted = obj.LBL1.Labels + .Where(z => !z.Name.EndsWith("_pl")) + .OrderBy(z => z.Index); + + foreach (var x in sorted) + { + var index = x.Index; + var name = x.Name; + var data = obj.TXT2.Strings[(int)index]; + var line = data.ToString(obj.FileEncoding).TrimEnd('\0'); + yield return $"{name}\t{line}"; + } + } + + public static IEnumerable GetOrderedLinesSingle(this MSBT obj) + { + var sorted = obj.LBL1.Labels + .Where(z => !z.Name.EndsWith("_pl")) + .OrderBy(z => z.Index); + + foreach (var x in sorted) + { + var index = x.Index; + var name = x.Name; + var data = obj.TXT2.Strings[(int)index]; + var line = data.ToString(obj.FileEncoding).TrimEnd('\0'); + yield return line; + } + } + } +} diff --git a/NewSnap.Lib/MSBT/TXT2.cs b/NewSnap.Lib/MSBT/TXT2.cs new file mode 100644 index 0000000..b55bedf --- /dev/null +++ b/NewSnap.Lib/MSBT/TXT2.cs @@ -0,0 +1,16 @@ +using System; +using System.Collections.Generic; + +namespace Lib +{ + public class TXT2 : MSBTSection + { + public uint NumberOfStrings; + + public readonly List Strings = new(); + + public TXT2() : base(string.Empty, Array.Empty()) + { + } + } +} diff --git a/NewSnap.Lib/Models/DrpArchive.cs b/NewSnap.Lib/Models/DrpArchive.cs new file mode 100644 index 0000000..d964037 --- /dev/null +++ b/NewSnap.Lib/Models/DrpArchive.cs @@ -0,0 +1,244 @@ +using System; +using System.Text; + +namespace NewSnap.Lib +{ + public class DrpArchive + { + /// Crc32(DRPF) + private const uint ArchiveHeaderMagic = 0x7F0E5359; + /// Crc32(fhdr) + private const uint CryptoBlockMagic = 0xC65753E8; + /// Crc32(resd) + private const uint FileBlockMagic = 0xE0A331B4; + /// Crc32(Oodl) + private const uint CompressedDataMagic = 0xE42D98BA; + + private readonly DrpFileEntry[] _files; + private readonly byte[] _seedTable = new byte[0x80]; + + public int FileCount => _files.Length; + + public string GetFileName(int i) => _files[i].GetFullFileName(); + public byte[] GetFileData(int i) => _files[i].Data; + + public DrpArchive(byte[] archive) + { + // Check the archive magic. + if (BitConverter.ToUInt32(archive, 0) != ArchiveHeaderMagic) + throw new ArgumentException("Invalid archive magic"); + + // Copy the encryption table. + Buffer.BlockCopy(archive, 0x14, _seedTable, 0, _seedTable.Length); + + // Decrypt the archive. + if (BitConverter.ToUInt32(archive, 8) != CryptoBlockMagic) + Decrypt(archive); + + // Verify archive header checksum (CRC32 over file count and seed table). + if (BitConverter.ToUInt32(archive, 4) != Crc32.ComputeChecksum(archive, 0x10, 0x84)) + throw new ArgumentException("Invalid archive checksum"); + + // Get the file count. + var fileCount = BitConverter.ToInt32(archive, 0x10); + _files = new DrpFileEntry[fileCount]; + + // Extract all files. + var offset = 0x94; + for (var i = 0; i < fileCount; ++i) + { + // Check the block magic. + if (BitConverter.ToUInt32(archive, offset + 4) != FileBlockMagic) + throw new ArgumentException("Invalid File Block Magic?"); + + // Get the chunk size. + var chunkSize = BitConverter.ToInt32(archive, offset + 8); + + // Read the data from the chunk + _files[i] = GetFile(archive, offset, chunkSize); + + // Advance to the next file. + offset += chunkSize; + } + } + + private void Decrypt(byte[] archive) + { + // Decrypt the crypto chunk (skipping data, as the crypto table isn't encrypted). + DecryptHeader(archive); + + // Verify the crypto table magic + if (BitConverter.ToUInt32(archive, 8) != CryptoBlockMagic) + throw new ArgumentException("Invalid Crypto Block Magic!"); + + // Verify the crypto table size + // TODO: There's logic for the case when this isn't 0x90 -- is this ever used? + if (BitConverter.ToUInt32(archive, 12) != 0x90) + throw new ArgumentException("Invalid Crypto Block Size!"); + + // TODO: Verify crypto block checksum + + // Get the file count. + var fileCount = BitConverter.ToInt32(archive, 16); + + // Decrypt each file block. + var offset = 0x94; + for (var i = 0; i < fileCount; ++i) + { + // Decrypt the current chunk. + DecryptChunk(archive, offset); + + // Check the chunk magic is what we expect. + // TODO: Is this guaranteed for all chunks in all archives? + if (BitConverter.ToUInt32(archive, offset + 4) != FileBlockMagic) + throw new ArgumentException("Invalid File Block Magic?"); + + // Advance + var chunkSize = BitConverter.ToInt32(archive, offset + 8); + offset += chunkSize; + } + + // Decrypt footer. + DecryptFooter(archive, offset); + } + + private void DecryptHeader(byte[] archive) + { + // Get the RNG + var seed = BitConverter.ToUInt32(archive, 4); + var rng = GetEncryptionRng(seed); + + // Decrypt the header + rng.DecryptWord(archive, 8); + rng.DecryptWord(archive, 12); + + // Decrypt the file count. + rng.DecryptWord(archive, 16); + } + + private void DecryptFooter(byte[] archive, int offset) + { + // Get the RNG + var seed = BitConverter.ToUInt32(archive, 4); + var rng = GetEncryptionRng(seed); + + // Decrypt to end. + while (offset < archive.Length) + { + rng.DecryptWord(archive, offset); + offset += sizeof(uint); + } + } + + private void DecryptChunk(byte[] archive, int offset, bool decryptData = true) + { + // Get the RNG + var seed = BitConverter.ToUInt32(archive, offset); + var rng = GetEncryptionRng(seed); + + // Decrypt the chunk header + rng.DecryptWord(archive, offset + 4); + rng.DecryptWord(archive, offset + 8); + + if (!decryptData) + return; + + // Decrypt the data + var chunkSize = BitConverter.ToUInt32(archive, offset + 8); + if ((chunkSize & 3) != 0) + throw new ArgumentException($"Invalid ChunkSize {chunkSize:X} at offset {offset:X}"); + + for (var i = 12; i < chunkSize; i += sizeof(uint)) + rng.DecryptWord(archive, offset + i); + } + + private XorShift GetEncryptionRng(uint seed) + { + var xs = GetXorshiftSeed(seed); + return new XorShift(xs); + } + + /// + /// The is interpreted as u8 indexes in the to build the actual seed. + /// + private uint GetXorshiftSeed(uint seed) + { + var key = 0u; + for (var i = 0; i < 4; ++i) + { + var index = (seed >> (i * 8)) & 0x7F; + key |= (uint)_seedTable[index] << (i * 8); + } + return key; + } + + private static DrpFileEntry GetFile(byte[] arc, int offset, int chunkSize) + { + //var fileBlockMagic = BitConverter.ToInt32(arc, offset + 0x04); + //var chunkSize = BitConverter.ToInt32(arc, offset + 0x08); + var extension = BitConverter.ToUInt32(arc, offset + 0x0C); + + // Get the file sizes. + var compressedSize = BitConverter.ToInt32(arc, offset + 0x10); + var decompressedSize = BitConverter.ToInt32(arc, offset + 0x14); + + // Get the file name length. + var fileNameLength = 0; + while (arc[offset + 0x18 + fileNameLength] != 0) + fileNameLength++; + + // Get the file name. + var fileName = Encoding.ASCII.GetString(arc, offset + 0x18, fileNameLength); + + // Extract the compressed data. + var compressedFileOffset = (0x18 + fileNameLength + 4) & ~3; + if (((compressedFileOffset + compressedSize + 3) & ~3) > chunkSize) + throw new ArgumentException($"Invalid chunk extents {compressedFileOffset:X} + {compressedSize:X} > {chunkSize:X}"); + + var compression = BitConverter.ToUInt32(arc, offset + compressedFileOffset); + if (compression == CompressedDataMagic) + { + // File is compressed. + var data = ReadCompressed(arc, offset, compressedSize, compressedFileOffset, decompressedSize); + return new DrpFileEntry(fileName, data, extension) {Compressed = true}; + } + else + { + // File is uncompressed. + if (compressedSize != decompressedSize) + throw new ArgumentException("Invalid uncompressed file extents"); + + var data = ReadDecompressed(arc, offset, compressedSize, compressedFileOffset); + return new DrpFileEntry(fileName, data, extension) {Compressed = false}; + } + } + + private static byte[] ReadDecompressed(byte[] arc, int offset, int compressedSize, int compressedFileOffset) + { + var result = new byte[compressedSize]; + Buffer.BlockCopy(arc, offset + compressedFileOffset, result, 0, result.Length); + return result; + } + + /// + /// Reads compressed data and decompresses it. + /// + /// Full Decrypted archive data + /// Start of the file's metadata + /// Length of compressed data + /// Offset where the compressed data begins + /// Length of data once decompressed. + /// Decompressed data + private static byte[] ReadCompressed(byte[] arc, int offset, int compressedSize, int compressedFileOffset, int decompressedSize) + { + var compressed = new byte[compressedSize - 4]; + Buffer.BlockCopy(arc, offset + compressedFileOffset + 4, compressed, 0, compressed.Length); + + // Decompress the file. + var decompressed = Oodle.Decompress(compressed, decompressedSize); + if (decompressed == null) + throw new ArgumentException("Failed to decompress file contents."); + return decompressed; + } + } +} diff --git a/NewSnap.Lib/Models/DrpFileEntry.cs b/NewSnap.Lib/Models/DrpFileEntry.cs new file mode 100644 index 0000000..ed16e4e --- /dev/null +++ b/NewSnap.Lib/Models/DrpFileEntry.cs @@ -0,0 +1,137 @@ +using System.Collections.Generic; +using System.Diagnostics; + +namespace NewSnap.Lib +{ + public class DrpFileEntry + { + /// + /// Relative path File Name + /// + /// Haven't seen any subfolder usage at this time. + public readonly string FileName; + + /// + /// Decrypted data for the file. + /// + public readonly byte[] Data; + + /// + /// Extension Magic (Crc32 encoded) + /// + /// + public readonly uint Extension; + + /// + /// When true, is stored inside a as a compressed bin. + /// + public bool Compressed { get; set; } + + /// + /// Indicates if the file name stored within the should keep the extension part of the string. + /// + /// Some files exclude the extension chars, while others retain it. + public bool HasExtensionFileName => FileName.Contains('.'); + + public DrpFileEntry(string fileName, byte[] data, uint extension) + { + Data = data; + FileName = fileName; + Extension = extension; + } + + /// + /// Gets the saved file name of the entry. + /// + public string GetFullFileName() + { + var ext = GetExtensionString(); + if (FileName.EndsWith(ext)) + return FileName; + if (HasExtensionFileName) + return FileName; + return FileName + ext; + } + + private string GetExtensionString() + { + var magic = Extension; + if (DrpFileExtensions.TryGetValue(magic, out var extension)) + return $".{extension}"; + + var result = magic.ToString("X8"); + Debug.WriteLine($"Unknown Extension CRC32: 0x{result}"); + return $".{result}"; + } + + /// + /// magic values used to store extension types. + /// + public static readonly IReadOnlyDictionary DrpFileExtensions = new Dictionary + { + {0x1C375F45, "txt"}, + {0xBE1C9ACB, "msbt"}, + {0xB9715ED2, "msbp"}, + {0xBB922BB1, "lm"}, + {0x5C156DBC, "nutexb"}, + {0x950C38A5, "bnk"}, + {0xCD63FEC8, "lmb"}, + {0x53076B6B, "lme"}, + {0x31C91A2C, "luo"}, + {0xA9B87C9F, "rtd"}, + {0xA76EEEC0, "shb"}, + {0x8C43BD03, "skb"}, + {0x0FB47EED, "achd"}, + {0x89E759E1, "allb"}, + {0xE42AB2FE, "cbsb"}, + {0xE165A47B, "cesb"}, + {0xB20291CC, "cutb"}, + {0x7A390130, "ecbd"}, + {0x9B70EEC4, "facb"}, + {0xF78548DD, "ldtb"}, + {0xBAF0E4F7, "lprb"}, + {0xA3D90E1B, "mdcd"}, + {0x37CD5F6B, "mdfb"}, + {0xEADAF976, "mdrp"}, + {0xE1C38F19, "misd"}, + {0x6971203F, "navb"}, + {0xA459AA15, "nvhb"}, + {0x0B548B0F, "path"}, + {0x58284CA4, "pcnb"}, + {0x01AA8159, "pdcd"}, + {0x6CD5ECCD, "pflb"}, + {0xB894D312, "pfrb"}, + {0x0972120A, "picd"}, + {0xF423150F, "pstb"}, + {0xBE2D954D, "ptsb"}, + {0x753C041E, "silb"}, + {0x97948F6D, "bfotf"}, + {0x87E7C3FC, "bfttf"}, + {0x2E9EDB6D, "efxbn"}, + {0x509961FC, "characterb"}, + {0x623D9D06, "matinstb"}, + {0xDAB89279, "numatb"}, + {0x8E6610FB, "modelb"}, + {0x0032C3E4, "nuanmb"}, + {0x81D2341C, "nuhlpb"}, + {0x236DB83A, "numshb"}, + {0x67E93703, "nusktb"}, + {0x3F86D270, "nusrcmdlb"}, + {0xC3157ABE, "courseb"}, + {0x9C5515DE, "nufxlb"}, + {0x2F6D9B0B, "nushdb"}, + {0x7E309203, "reflectb"}, + {0xE57E2E01, "paramb"}, + {0x2E641827, "stfrolb"}, + {0xE9AB884F, "genderb"}, + {0xF59A44EB, "levelb"}, + {0x62BC395A, "navmshb"}, + + // Unknown: Probably long extensions. If ya know the real one, pull request! + // {0xDC4A8177, "DC4A8177"}, // within romfs + // {0xC63E569C, "C63E569C"}, // within savedata + // {0xB519FC35, "B519FC35"}, // within savedata + // {0x9B7A0B7C, "9B7A0B7C"}, // within savedata + }; + } +} diff --git a/NewSnap.Lib/NewSnap.Lib.csproj b/NewSnap.Lib/NewSnap.Lib.csproj new file mode 100644 index 0000000..ec8c055 --- /dev/null +++ b/NewSnap.Lib/NewSnap.Lib.csproj @@ -0,0 +1,9 @@ + + + + netcoreapp3.1 + 9 + enable + + + diff --git a/NewSnap.Lib/Saves/SaveDumper.cs b/NewSnap.Lib/Saves/SaveDumper.cs new file mode 100644 index 0000000..64c724e --- /dev/null +++ b/NewSnap.Lib/Saves/SaveDumper.cs @@ -0,0 +1,79 @@ +using System; +using System.IO; + +namespace NewSnap.Lib +{ + public static class SaveDumper + { + public static void ExtractEntries(string srcDir, string destDir) + { + var files = SaveReader.SaveFileNames; + for (var index = 0; index < files.Count; ++index) + ExtractEntries(srcDir, destDir, index); + } + + private static void ExtractEntries(string srcDir, string destDir, int index) + { + var files = SaveReader.SaveFileNames; + var file = Path.Combine(srcDir, files[index]); + ExtractEntries(file, index, destDir); + } + + private static void ExtractEntries(string file, int index, string destDir) + { + if (!File.Exists(file)) + return; + + var subDir = Path.Combine(destDir, $"{index:00}_out"); + Directory.CreateDirectory(subDir); + + var stream = File.OpenRead(file); + using var sav = new SaveFile(stream, index); + + // For jpeg archives, there's a lot of files. + // Sorting by filename is good, so pad with zeroes. + var count = sav.EntryCount; + var digits = (int) Math.Floor(Math.Log10(count) + 1); + + for (int i = 0; i < count; i++) + { + var data = sav.GetEntry(i); + if (data.Length == 0) + continue; + + var ext = "bin"; + if (IsJpegData(data)) + { + ext = "jpg"; + data = GetJpegOnly(data, 0x10); + } + + var fn = i.ToString().PadLeft(digits, '0'); + var dest = Path.Combine(subDir, $"{fn}.{ext}"); + File.WriteAllBytes(dest, data); + } + } + + private static bool IsJpegData(byte[] data) + { + // First 0x10 bytes are metadata we don't care about. + if (data.Length <= 0x20) + return false; + if (BitConverter.ToUInt16(data, 0x10) != 0xD8FF) // SOI + return false; + if (BitConverter.ToUInt32(data, 0x16) != 0x4649464A) // JFIF + return false; + return true; + } + + private static byte[] GetJpegOnly(byte[] data, int start) + { + Span end = stackalloc byte[] { 0xFF, 0xD9 }; + var x = data.AsSpan(); + var length = x.LastIndexOf(end); + if (length == -1) + throw new ArgumentException("No EOF found inside jpeg data!"); + return data[start..(length + 2)]; + } + } +} diff --git a/NewSnap.Lib/Saves/SaveFile.cs b/NewSnap.Lib/Saves/SaveFile.cs new file mode 100644 index 0000000..8a6a54f --- /dev/null +++ b/NewSnap.Lib/Saves/SaveFile.cs @@ -0,0 +1,75 @@ +using System; +using System.Diagnostics; +using System.IO; + +namespace NewSnap.Lib +{ + public class SaveFile : IDisposable + { + private readonly Stream Stream; + private readonly SaveFileHeader Header; + private readonly int SaveIndex; + + public int EntryCount => Header.EntryCount; + + public SaveFile(Stream stream, int index) + { + Stream = stream; + SaveIndex = index; + stream.Position = 0; + Span hdr = stackalloc byte[0x30]; + stream.Read(hdr); + + Header = SaveReader.GetSaveFileHeader(hdr, index); + } + + public void Dispose() + { + Dispose(true); + GC.SuppressFinalize(this); + } + + private bool isDisposed; + + protected virtual void Dispose(bool disposing) + { + if (isDisposed) + return; + + if (disposing) + { + try + { + Stream.Close(); + Stream.Dispose(); + } +#pragma warning disable CA1031 // Do not catch general exception types + catch (Exception x) +#pragma warning restore CA1031 // Do not catch general exception types + { + Debug.WriteLine($"Already closed? {x}"); + } + } + isDisposed = true; + } + + public byte[] GetEntry(int index) + { + if ((uint) index >= Header.EntryCount) + throw new ArgumentOutOfRangeException(nameof(index)); + + var size = Header.EncryptedEntrySize; + var start = 0x30 + (index * size); + + Span encEntry = stackalloc byte[size]; + Stream.Position = start; + Stream.Read(encEntry); + + if (BitConverter.ToUInt64(encEntry) == 0) + return Array.Empty(); + + var e = SaveReader.GetSaveFileEntry(encEntry, SaveIndex, index); + return e.Data[..Header.DecryptedEntrySize]; + } + } +} diff --git a/NewSnap.Lib/Saves/SaveFileEntry.cs b/NewSnap.Lib/Saves/SaveFileEntry.cs new file mode 100644 index 0000000..fc805b6 --- /dev/null +++ b/NewSnap.Lib/Saves/SaveFileEntry.cs @@ -0,0 +1,33 @@ +using System; + +namespace NewSnap.Lib +{ + public class SaveFileEntry + { + public uint AdlerChecksum; + public uint Crc32Checksum; + public uint Magic; + public int DataSize; + public int DataSizeDuplicate; + public uint Unk14; + public uint Unk18; + public uint Unk1C; + + public byte[] Data; + + public SaveFileEntry(ReadOnlySpan decEntry) + { + AdlerChecksum = BitConverter.ToUInt32(decEntry); + Crc32Checksum = BitConverter.ToUInt32(decEntry[4..]); + Magic = BitConverter.ToUInt32(decEntry[8..]); + DataSize = BitConverter.ToInt32(decEntry[0xC..]); + DataSizeDuplicate = BitConverter.ToInt32(decEntry[0x10..]); + + Unk14 = BitConverter.ToUInt32(decEntry[0x14..]); + Unk18 = BitConverter.ToUInt32(decEntry[0x18..]); + Unk1C = BitConverter.ToUInt32(decEntry[0x1C..]); + + Data = decEntry[0x20..].ToArray(); + } + } +} diff --git a/NewSnap.Lib/Saves/SaveFileHeader.cs b/NewSnap.Lib/Saves/SaveFileHeader.cs new file mode 100644 index 0000000..3219600 --- /dev/null +++ b/NewSnap.Lib/Saves/SaveFileHeader.cs @@ -0,0 +1,29 @@ +using System; + +namespace NewSnap.Lib +{ + public class SaveFileHeader + { + public uint Magic; + public int EncryptedEntrySize; + public int DecryptedEntrySize; + public int EntryCount; + public int SizeMaybe; + public uint Unk14; + public uint Unk18; + public uint Unk1C; + + public SaveFileHeader(ReadOnlySpan decHeader) + { + Magic = BitConverter.ToUInt32(decHeader); + EncryptedEntrySize = BitConverter.ToInt32(decHeader[4..]); + DecryptedEntrySize = BitConverter.ToInt32(decHeader[8..]); + EntryCount = BitConverter.ToInt32(decHeader[0xC..]); + SizeMaybe = BitConverter.ToInt32(decHeader[0x10..]); + + Unk14 = BitConverter.ToUInt32(decHeader[0x14..]); + Unk18 = BitConverter.ToUInt32(decHeader[0x18..]); + Unk1C = BitConverter.ToUInt32(decHeader[0x1C..]); + } + } +} diff --git a/NewSnap.Lib/Saves/SaveReader.cs b/NewSnap.Lib/Saves/SaveReader.cs new file mode 100644 index 0000000..634c3c0 --- /dev/null +++ b/NewSnap.Lib/Saves/SaveReader.cs @@ -0,0 +1,136 @@ +using System; +using System.Collections.Generic; +using System.IO; +using System.Linq; +using System.Security.Cryptography; + +namespace NewSnap.Lib +{ + public static class SaveReader + { + public static readonly IReadOnlyList SaveFileNames = new[] + { + "5652712535912f19", + "398ecf2b56c614e3", + "6706e7d0f3dab2ab", + "00c31e53dde3ac51", + "4f342de95be7450e", + "0e7280b1c3ce5a0c", + "04289da492638279", + "9c1c815c54797c18", + "bf0b4b223f011bf9", + "4b1a0e859a3f7816", + "b8b0ce894762e7a8", + "76401e93e6565bd5", + "0a980178a54e26f3", + "4207cfe275441231", + "75006a27cf0c9275", + "9661edcf41d19929", + }; + + public static readonly IReadOnlyList SaveFileKeys = new[] + { + 0xB519FC35, + 0xB519FC36, + 0xB519FC37, + 0xB519FC38, + 0xB519FC39, + 0xB519FC3A, + 0xB519FC3B, + 0xB519FC3C, + 0xB519FC3D, + 0xB519FC3E, + 0xB519FC3F, + 0xB519FC40, + 0xB519FC41, + 0xB519FC42, + 0xB519FC43, + 0xB519FC44, + }; + + public static bool IsCompleteSaveDirectory(string dir) => Directory.EnumerateFiles(dir).All(SaveFileNames.Contains); + + public static readonly byte[] HeaderKey = + { + 0x1F, 0xC5, 0xD5, 0x71, 0xBD, 0xEF, 0xAF, 0x83, 0xFC, 0x96, 0xEE, 0xFE, 0x70, 0xA1, 0x14, 0xEC + }; + + /// + /// Gets a from the input save file's raw header data. + /// + /// Header data + /// Save File Index + public static SaveFileHeader GetSaveFileHeader(ReadOnlySpan header, int index) + { + var seed = SaveFileKeys[index]; + var key = HeaderKey; + + var decHeader = DecryptHeader(header, seed, key); + return new SaveFileHeader(decHeader); + } + + /// + /// Gets a from the input save file's raw entry data. + /// + /// Entry data + /// Save File Index + /// File Index within Save File + public static SaveFileEntry GetSaveFileEntry(ReadOnlySpan entry, int index, int entryIndex) + { + var seed = SaveFileKeys[index]; + var key = HeaderKey; + + var decEntry = DecryptEntry(entry, seed + (uint)entryIndex, key); + return new SaveFileEntry(decEntry); + } + + public static byte[] DecryptHeader(ReadOnlySpan header, uint seed, byte[] key) + { + var rng = new XorShift(seed); + var dat = header[..0x20]; + var encIV = header[0x20..0x30]; + + var iv = ReadDecryptedIV(encIV, rng); + return Decrypt(key, iv, dat); + } + + private static byte[] DecryptEntry(ReadOnlySpan entry, uint seed, byte[] key) + { + var rng = new XorShift(seed); + var encIV = entry[..0x10]; + var dat = entry[0x10..]; + + var iv = ReadDecryptedIV(encIV, rng); + return Decrypt(key, iv, dat); + } + + private static byte[] Decrypt(byte[] key, byte[] iv, ReadOnlySpan dat) + { + using var ms = new MemoryStream(dat.Length); + ms.Write(dat); + ms.Position = 0; + + using var aes = new AesManaged + { + Key = key, + IV = iv, + Mode = CipherMode.CBC, + Padding = PaddingMode.None + }; + using var decryptor = aes.CreateDecryptor(); + using var cs = new CryptoStream(ms, decryptor, CryptoStreamMode.Read); + + var decrypted = new byte[dat.Length]; + cs.Read(decrypted); + return decrypted; + } + + private static byte[] ReadDecryptedIV(ReadOnlySpan entry, XorShift rng) + { + var iv = new byte[0x10]; + for (var i = 0x0; i < iv.Length; ++i) + iv[i] = (byte)(entry[i] ^ (rng.GetNext(0xFE) + 1)); + return iv; + } + } +} diff --git a/NewSnap.Tests/CrcTests.cs b/NewSnap.Tests/CrcTests.cs new file mode 100644 index 0000000..f28fe03 --- /dev/null +++ b/NewSnap.Tests/CrcTests.cs @@ -0,0 +1,25 @@ +using FluentAssertions; +using NewSnap.Lib; +using Xunit; + +namespace NewSnap.Tests +{ + public class CrcTests + { + [Fact] + public void ExtensionTests() + { + var dict = DrpFileEntry.DrpFileExtensions; + foreach (var (key, ext) in dict) + TestCrcString(ext, key); + } + + [Theory] + [InlineData("nutexb", 0x5C156DBC)] + public void TestCrcString(string str, uint expect) + { + var crc = Crc32.ComputeChecksum(str); + crc.Should().Be(expect); + } + } +} diff --git a/NewSnap.Tests/MSBTTests.cs b/NewSnap.Tests/MSBTTests.cs new file mode 100644 index 0000000..80a7510 --- /dev/null +++ b/NewSnap.Tests/MSBTTests.cs @@ -0,0 +1,41 @@ +using System.IO; +using Lib; +using Xunit; + +namespace NewSnap.Tests +{ + public class MSBTTests + { + private const string path = @"E:\snapdump\"; + private const string outDir = @"E:\snapdump_text\"; + + [Fact] + public void DumpAllMSBT() + { + var files = Directory.GetFiles(path, "*.msbt", SearchOption.AllDirectories); + foreach (var f in files) + { + var data = File.ReadAllBytes(f); + var msbt = new MSBT(data); + DumpMSBT(f, msbt); + } + } + + private static void DumpMSBT(string f, MSBT msbt) + { + var destPath = f.Replace(path, outDir); + var dir = new FileInfo(destPath).Directory.FullName; + Directory.CreateDirectory(dir); + + var file = Path.Combine(dir, Path.GetFileNameWithoutExtension(destPath)); + var x = msbt.GetOrderedLines(); + File.WriteAllLines(file + "_raw.txt", x); + + var y = msbt.GetOrderedLinesTab(); + File.WriteAllLines(file + "_tab.txt", y); + + var z = msbt.GetOrderedLinesSingle(); + File.WriteAllLines(file + ".txt", z); + } + } +} diff --git a/NewSnap.Tests/NewSnap.Tests.csproj b/NewSnap.Tests/NewSnap.Tests.csproj new file mode 100644 index 0000000..4315777 --- /dev/null +++ b/NewSnap.Tests/NewSnap.Tests.csproj @@ -0,0 +1,25 @@ + + + + netcoreapp3.1 + 9 + enable + + false + + + + + + + + runtime; build; native; contentfiles; analyzers; buildtransitive + all + + + + + + + + diff --git a/NewSnap.Tests/SaveTests.cs b/NewSnap.Tests/SaveTests.cs new file mode 100644 index 0000000..fde71ff --- /dev/null +++ b/NewSnap.Tests/SaveTests.cs @@ -0,0 +1,124 @@ +using System; +using System.IO; +using FluentAssertions; +using NewSnap.Lib; +using Xunit; + +namespace NewSnap.Tests +{ + public class SaveTests + { + [Fact] + public void DumpSaveNames() + { + const ulong lo = 0x2163A6B38429BC22UL; + const ulong hi = 0x9B4923E9F5AAB470UL; + var rng = new XorShift(lo, hi); + for (int i = 0; i < 15; i++) + { + var first = rng.GetNext(uint.MaxValue); + var second = rng.GetNext(uint.MaxValue); + var fn = $"{first:x8}{second:x8}"; + var expect = SaveReader.SaveFileNames[i]; + fn.Should().Be(expect); + } + } + + [Fact] + public void TestHeaderDecrypt() + { + var h = SaveReader.GetSaveFileHeader((byte[])Header00C_3.Clone(), 3); + + h.Magic.Should().Be(0xC63E569C); + h.EncryptedEntrySize.Should().Be(0x70840); + h.DecryptedEntrySize.Should().Be(0x70810); + h.SizeMaybe.Should().Be(0x30); + } + + [Fact] + public void TestEntryDecrypt() + { + var e = SaveReader.GetSaveFileEntry((byte[])Entry000_0.Clone(), 0, 0); + + e.Magic.Should().Be(0x9B7A0B7C); + e.AdlerChecksum.Should().Be(Adler32.ComputeChecksum(e.Data)); + e.Crc32Checksum.Should().Be(Crc32.ComputeChecksum(e.Data)); + e.DataSize.Should().Be(e.Data.Length); + e.DataSizeDuplicate.Should().Be(e.Data.Length); + } + + private static readonly byte[] Header00C_3 = + { + 0x50, 0x70, 0x20, 0x84, 0x64, 0xD6, 0x19, 0xBE, 0x34, 0x93, 0xE0, 0x3B, 0x05, 0x04, 0xDD, 0xD8, + 0x98, 0xB0, 0x9B, 0xE4, 0xCF, 0xE5, 0xA1, 0x4E, 0xAC, 0xA3, 0x94, 0xA4, 0xBA, 0x71, 0x13, 0x3F, + 0xCC, 0xC6, 0xE8, 0x77, 0xA8, 0x9C, 0xCC, 0xB1, 0x94, 0xBC, 0x4F, 0xDD, 0xA7, 0x7F, 0x44, 0x00, + }; + + private static readonly byte[] Entry000_0 = + { + 0x85, 0x11, 0xDD, 0x8E, 0xCE, 0xD4, 0x3A, 0x3B, 0x73, 0x1C, 0x49, 0x7A, 0x75, 0xEB, 0xAB, 0xEA, + 0xEF, 0x6F, 0x31, 0x54, 0x45, 0xA1, 0xD4, 0xA1, 0xF9, 0x40, 0x7E, 0x7E, 0x14, 0xAB, 0x62, 0x89, + 0xA1, 0xF9, 0x41, 0xE6, 0x57, 0xB5, 0x9D, 0xCC, 0x8B, 0xA9, 0xD5, 0x0B, 0x55, 0xB1, 0x0B, 0x64, + 0xDB, 0x9B, 0xCB, 0xB4, 0x97, 0x48, 0xF3, 0x6B, 0x6C, 0xA5, 0x31, 0xC5, 0x0B, 0x7D, 0x04, 0x34, + 0xB9, 0x47, 0x7D, 0x6C, 0xA9, 0xD1, 0x2E, 0x45, 0x71, 0xC5, 0x8C, 0x79, 0x72, 0xAB, 0x30, 0x4E, + 0xCC, 0x0A, 0x42, 0x2A, 0x47, 0xC9, 0x2A, 0xDE, 0x63, 0x86, 0x5F, 0x44, 0x95, 0xC0, 0x2D, 0xCE, + 0x9B, 0xAC, 0x99, 0xE6, 0xA6, 0x6B, 0x1F, 0xA2, 0x56, 0x10, 0x8F, 0x5A, 0xEB, 0xA9, 0xE9, 0x47, + 0x75, 0x64, 0x61, 0xF8, 0x78, 0x22, 0xA9, 0x20, 0x21, 0x12, 0xE2, 0x29, 0x0E, 0x08, 0x78, 0xD5, + 0x3E, 0x56, 0x9C, 0xF2, 0x7B, 0x36, 0xF8, 0x89, 0x5E, 0x72, 0x9D, 0x88, 0xCC, 0x0B, 0x4E, 0x65, + }; + + [Fact] + public void TestDecryptFull() + { + const string dir = @"E:\snap\save\"; + for (var index = 0; index < 16; ++index) + { + var name = SaveReader.SaveFileNames[index]; + var file = Path.Combine(dir, name); + if (!File.Exists(file)) + continue; + + var encsave = File.ReadAllBytes(file); + + var h = SaveReader.GetSaveFileHeader(new ReadOnlySpan(encsave, 0, 0x30), index); + + var decsave = new byte[0x20 + (h.EntryCount * h.DecryptedEntrySize)]; + var encHeader = encsave.AsSpan(..0x30); + SaveReader.DecryptHeader(encHeader, SaveReader.SaveFileKeys[index], SaveReader.HeaderKey).CopyTo(decsave, 0); + + for (var i = 0; i < h.EntryCount; ++i) + { + var start = 0x30 + (i * h.EncryptedEntrySize); + if (BitConverter.ToUInt64(encsave, start) == 0) + continue; + + var encEntry = encsave.AsSpan(start, h.EncryptedEntrySize); + + var e = SaveReader.GetSaveFileEntry(encEntry, index, i); + + var region = e.Data.AsSpan(..h.DecryptedEntrySize); + + e.AdlerChecksum.Should().Be(Adler32.ComputeChecksum(region)); + e.Crc32Checksum.Should().Be(Crc32.ComputeChecksum(region)); + e.Magic.Should().Be(0x9B7A0B7C); + e.DataSize.Should().BeLessOrEqualTo(e.Data.Length); + e.DataSizeDuplicate.Should().BeLessOrEqualTo(e.Data.Length); + e.Data.Length.Should().Be((h.DecryptedEntrySize + 15) & ~15); + + var decDest = decsave.AsSpan(0x20 + (i * h.DecryptedEntrySize)); + region.CopyTo(decDest); + } + + var dest = Path.Combine(dir, $"{name}.dec"); + File.WriteAllBytes(dest, decsave); + } + } + + [Fact] + public void TestExtractFull() + { + const string srcDir = @"E:\snap\save\"; + SaveDumper.ExtractEntries(srcDir, srcDir); + } + } +} diff --git a/NewSnap.sln b/NewSnap.sln new file mode 100644 index 0000000..4a5949e --- /dev/null +++ b/NewSnap.sln @@ -0,0 +1,37 @@ + +Microsoft Visual Studio Solution File, Format Version 12.00 +# Visual Studio Version 16 +VisualStudioVersion = 16.0.31005.135 +MinimumVisualStudioVersion = 10.0.40219.1 +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "NewSnap.Lib", "NewSnap.Lib\NewSnap.Lib.csproj", "{4126F2B1-C2C4-42DF-A288-31400FA3B01D}" +EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "NewSnap.Tests", "NewSnap.Tests\NewSnap.Tests.csproj", "{8B5B295A-74AE-4480-99A9-ADC5C604E7D1}" +EndProject +Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "NewSnap.App", "NewSnap.App\NewSnap.App.csproj", "{7B42A91F-4EE7-48D1-B759-A819DB3061C5}" +EndProject +Global + GlobalSection(SolutionConfigurationPlatforms) = preSolution + Debug|Any CPU = Debug|Any CPU + Release|Any CPU = Release|Any CPU + EndGlobalSection + GlobalSection(ProjectConfigurationPlatforms) = postSolution + {4126F2B1-C2C4-42DF-A288-31400FA3B01D}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {4126F2B1-C2C4-42DF-A288-31400FA3B01D}.Debug|Any CPU.Build.0 = Debug|Any CPU + {4126F2B1-C2C4-42DF-A288-31400FA3B01D}.Release|Any CPU.ActiveCfg = Release|Any CPU + {4126F2B1-C2C4-42DF-A288-31400FA3B01D}.Release|Any CPU.Build.0 = Release|Any CPU + {8B5B295A-74AE-4480-99A9-ADC5C604E7D1}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {8B5B295A-74AE-4480-99A9-ADC5C604E7D1}.Debug|Any CPU.Build.0 = Debug|Any CPU + {8B5B295A-74AE-4480-99A9-ADC5C604E7D1}.Release|Any CPU.ActiveCfg = Release|Any CPU + {8B5B295A-74AE-4480-99A9-ADC5C604E7D1}.Release|Any CPU.Build.0 = Release|Any CPU + {7B42A91F-4EE7-48D1-B759-A819DB3061C5}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {7B42A91F-4EE7-48D1-B759-A819DB3061C5}.Debug|Any CPU.Build.0 = Debug|Any CPU + {7B42A91F-4EE7-48D1-B759-A819DB3061C5}.Release|Any CPU.ActiveCfg = Release|Any CPU + {7B42A91F-4EE7-48D1-B759-A819DB3061C5}.Release|Any CPU.Build.0 = Release|Any CPU + EndGlobalSection + GlobalSection(SolutionProperties) = preSolution + HideSolutionNode = FALSE + EndGlobalSection + GlobalSection(ExtensibilityGlobals) = postSolution + SolutionGuid = {55C8F5E2-4981-4898-9281-DD1366215B3A} + EndGlobalSection +EndGlobal diff --git a/README.md b/README.md new file mode 100644 index 0000000..7093182 --- /dev/null +++ b/README.md @@ -0,0 +1,19 @@ +NewSnap +===== +![License](https://img.shields.io/badge/License-ISC-blue.svg?style=flat-square) + +New Pokémon Snap class libary and console application programmed in [C#](https://en.wikipedia.org/wiki/C_Sharp_%28programming_language%29). + +Supports handling of `*.drp` archives (from the ROM/patches) and unpacking data from save files. + +The console application provides a way to manually execute preprogrammed routines to export your game data. + +## Building + +NewSnap is a [.NET Core 3.1](https://dotnet.microsoft.com/download/dotnet/3.1) project which can be run on Windows/Mac/Linux. + +The solution can be built with any compiler that supports **C# 9**. We recommend using IDEs such as [Visual Studio](https://visualstudio.microsoft.com/downloads/) or [Rider](https://www.jetbrains.com/rider/download/). They can open the .sln or .csproj file. + +## Dependencies + +Decompressing data within a `*.drp` archive requires having the [Oodle Decompressor dll](http://www.radgametools.com/oodlecompressors.htm) in the same folder as the executable. This program has a hardcoded reference to `oo2core_8_win64.dll`, which can be sourced from other games (for example Warframe, which is free on Steam).