Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support for compact tool (WofCompressionData) #8

Closed
vGimly opened this issue Feb 11, 2022 · 3 comments
Closed

Support for compact tool (WofCompressionData) #8

vGimly opened this issue Feb 11, 2022 · 3 comments
Assignees
@thewhiteninja
Labels
enhancement New feature or request

Comments

@vGimly
Copy link
Contributor

vGimly commented Feb 11, 2022 •
edited
Loading

Windows 10 introduces Windows Overlay Filtering compression for NTFS.
And there even Windows 10 "Compact Edition", where that compression is turned on by default.

To check it just run in windows 10/11:

compact.exe /c /exe:lzx c:\windows\splwow64.exe
ntfstool extract disk=1 volume=2 from="c:\windows\splwow64.exe:WofCompressedData" output="c:\splwow64.exe,compact"

Note sparse/reparse flags are set, new ADS $DATA:WofCompressedData, $REPARSE_POINT with Type "Windows Overlay" IO_REPARSE_TAG_WOF .

MFT (inode:396322) from \\.\PhysicalDrive1 > Volume:2
-----------------------------------------------------

Signature         : FILE
Update Offset     : 48
Update Number     : 3
$LogFile LSN      : 285433403226
Sequence Number   : 1
Hardlink Count    : 2
Attribute Offset  : 56
Flags             : In_use
Real Size         : 936
Allocated Size    : 1024
Base File Record  : 0000000000000000h
Next Attribute ID : 14
MFT Record Index  : 396322
Update Seq Number : 8
Update Seq Array  : 00000000

Attributes:
-----------

+------------------------------------------------------------------------------------------------------+
| Id | Type                   | Non-resident | Length | Overview                                       |
+------------------------------------------------------------------------------------------------------+
| 1  | $STANDARD_INFORMATION  | False        | 72     | File Created Time       : 2020-09-02 12:49:59  |
|    |                        |              |        | Last File Write Time    : 2022-02-12 02:14:21  |
|    |                        |              |        | FileRecord Changed Time : 2022-02-12 02:14:21  |
|    |                        |              |        | Last Access Time        : 2022-02-12 02:14:21  |
|    |                        |              |        | Permissions             :                      |
|    |                        |              |        |   read_only     : 0                            |
|    |                        |              |        |   hidden        : 0                            |
|    |                        |              |        |   system        : 0                            |
|    |                        |              |        |   device        : 0                            |
|    |                        |              |        |   normal        : 0                            |
|    |                        |              |        |   temporary     : 0                            |
|    |                        |              |        |   sparse        : 1                            |
|    |                        |              |        |   reparse_point : 1                            |
|    |                        |              |        |   compressed    : 0                            |
|    |                        |              |        |   offline       : 0                            |
|    |                        |              |        |   not_indexed   : 0                            |
|    |                        |              |        |   encrypted     : 0                            |
|    |                        |              |        | Max Number of Versions  : 0                    |
|    |                        |              |        | Version Number          : 0                    |
+------------------------------------------------------------------------------------------------------+
| 2  | $FILE_NAME             | False        | 90     | Parent Dir Record Index : 396463               |
|    |                        |              |        | Parent Dir Sequence Num : 1                    |
|    |                        |              |        | File Created Time       : 2020-09-02 12:49:59  |
|    |                        |              |        | Last File Write Time    : 2020-09-02 12:49:59  |
|    |                        |              |        | FileRecord Changed Time : 2020-09-02 12:49:59  |
|    |                        |              |        | Last Access Time        : 2020-09-02 12:49:59  |
|    |                        |              |        | Allocated Size          : 167936               |
|    |                        |              |        | Real Size               : 165376               |
|    |                        |              |        | ------                                         |
|    |                        |              |        | NameType                : DOS & WIN32          |
|    |                        |              |        | Name                    : splwow64.exe         |
+------------------------------------------------------------------------------------------------------+
| 3  | $FILE_NAME             | False        | 90     | Parent Dir Record Index : 493                  |
|    |                        |              |        | Parent Dir Sequence Num : 1                    |
|    |                        |              |        | File Created Time       : 2020-09-02 12:49:59  |
|    |                        |              |        | Last File Write Time    : 2020-09-02 17:31:19  |
|    |                        |              |        | FileRecord Changed Time : 2020-09-02 12:49:59  |
|    |                        |              |        | Last Access Time        : 2020-09-02 12:49:59  |
|    |                        |              |        | Allocated Size          : 167936               |
|    |                        |              |        | Real Size               : 165376               |
|    |                        |              |        | ------                                         |
|    |                        |              |        | NameType                : POSIX                |
|    |                        |              |        | Name                    : splwow64.exe         |
+------------------------------------------------------------------------------------------------------+
| 4  | $DATA                  | True         | 165376 | Data Size               : 165376 (161.50 KiBs) |
|    |                        |              |        | Flags                   :                      |
|    |                        |              |        |     Sparse                                     |
|    |                        |              |        | Dataruns                :                      |
|    |                        |              |        |     Length: 00000030 Offset: 00000000 (S)      |
|    |                        |              |        | Size on disk            : 0 (0.00 byte)        |
+------------------------------------------------------------------------------------------------------+
| 5  | $DATA                  | True         | 88570  | Name                    : WofCompressedData    |
|    |                        |              |        | Data Size               : 88570 (86.49 KiBs)   |
|    |                        |              |        | Dataruns                :                      |
|    |                        |              |        |     Length: 00000016 Offset: 031e33bc          |
|    |                        |              |        | Size on disk            : 90112 (88.00 KiBs)   |
+------------------------------------------------------------------------------------------------------+
| 6  | $REPARSE_POINT         | False        | 24     | Type                    : Windows Overlay      |
|    |                        |              |        | Unsupported reparse point type                 |
+------------------------------------------------------------------------------------------------------+
| 7  | $EA_INFORMATION        | False        | 8      | NYI attribute type                             |
+------------------------------------------------------------------------------------------------------+
| 8  | $EA                    | False        | 92     | NYI attribute type                             |
+------------------------------------------------------------------------------------------------------+
| 9  | $LOGGED_UTILITY_STREAM | False        | 8      | Binary data                                    |
+------------------------------------------------------------------------------------------------------+
| 10 | $LOGGED_UTILITY_STREAM | False        | 56     | Binary data                                    |
+------------------------------------------------------------------------------------------------------+

There is XPRESS4K/XPRES8K/XPRESS16K/LZX compression algo. And also this tag should support for WIM files (reparse point can reference to WIM archive) - but I have no proof for it.
https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/compact-os?view=windows-11
RtlDecompressBufferEx have support for COMPRESSION_FORMAT_XPRESS_HUFF - for WIM-LZX decompressor can be found on github...
@thewhiteninja thewhiteninja self-assigned this Feb 13, 2022
@thewhiteninja thewhiteninja added the enhancement New feature or request label Feb 13, 2022
@vGimly
Copy link
Contributor Author

vGimly commented Feb 16, 2022 •
edited
Loading

$REPARSE_POINT attribute for IO_REPARSE_TAG_WOF:
length is always 16 bytes (4 dwords): 1,2,1,PACK_TYPE

PACK_TYPE = AlgName (BLOCK_SIZE)
0 = XPRESS4K
1 = LZX (32k)
2 = XPRESS8K
3 = XPRESS16K

eg $REPARSE_POINT data:
01 00 00 00 02 00 00 00 01 00 00 00 02 00 00 00 = XPRESS8K
01 00 00 00 02 00 00 00 01 00 00 00 01 00 00 00 = LZX

WofCompressedData stream consists in two parts:

  1. (optional) DWORD offsets to fragments.
  2. compressed fragment data.

Compression logic:

  1. File splits into fragments of BLOCK_SIZE. If file size lower or equals size of the single fragment, there is no offsets table in the beginning of the stream.
  2. Each fragment compressed/uncompressed independently.

Compression/decompression with ntdll (only XPRESS, no LZX):

DWORD64 comp_block, comp_frag;
RtlGetCompressionWorkSpaceSize(COMPRESSION_FORMAT_XPRESS_HUFF, &comp_block, &comp_frag);
LPVOID workspace = LocalAlloc(LMEM_FIXED, comp_block);
RtlCompressBuffer(COMPRESSION_FORMAT_XPRESS_HUFF, uncomp, uncomp_len, comp, comp_buf_len, block_size, &packed, workspace);
RtlDecompressBufferEx(COMPRESSION_FORMAT_XPRESS_HUFF, uncomp, uncomp_buf_len, comp,comp_len, &unpacked, workspace);
LocalFree(workspace);

eg for XPRESS16K, source file 32k of 00 bytes:

0000: 07 01 00 00 *02 00 00 00 │ 00 00 00 00 00 00 00 00
0010: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0020: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0030: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0040: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0050: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0060: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0070: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0080: 00 00 00 00 02 00 00 00 │ 00 00 00 10 00 00 00 00
0090: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
00A0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
00B0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
00C0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
00D0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
00E0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
00F0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0100: 00 00 00 00 00 98 00 00 │ FF FC 3F *02 00 00 00 00
0110: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0120: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0130: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0140: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0150: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0160: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0170: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0180: 00 00 00 00 00 00 00 00 │ 00 00 00 02 00 00 00 00
0190: 00 00 10 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
01A0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
01B0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
01C0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
01D0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
01E0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
01F0: 00 00 00 00 00 00 00 00 │ 00 00 00 00 00 00 00 00
0200: 00 00 00 00 00 00 00 00 │ 00 00 00 00 98 00 00 FF
0210: FC 3F

Two fragments (32K/16K), so offsets table have 1 element (0x107).
First fragment offset 0x04, length 0x107 (due to offsets table),
Second fragment offset 0x107+0x04 = 0x10B, length till the end of the stream.

@thewhiteninja
Copy link
Owner

Thanks for the details!
I have a working PoC for xpress, need to clean the code a bit and I will push it

@thewhiteninja
Copy link
Owner

With ed083d4, you should be able to extract wof (xpress) files using:
ntfstool extract disk=1 volume=2 from="c:\windows\splwow64.exe:WofCompressedData" output="c:\splwow64.exe"

I also added description for $REPARSE_POINT & wof
Ex:

+----------------------------------------------------------------------------------------------------------+
| 5  | $REPARSE_POINT             | False        | 24      | Type                    : Windows Overlay     |
|    | Raw address: 014516e7f1d8h |              |         | ------                                        |
|    |                            |              |         | Version                 : 1                   |
|    |                            |              |         | Provider                : 2                   |
|    |                            |              |         | File Version            : 1                   |
|    |                            |              |         | Compression Algorithm   : LZX 32k             |
+----------------------------------------------------------------------------------------------------------+

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@thewhiteninja thewhiteninja

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vGimly @thewhiteninja