ecm new exeflat format #78
Conversation
This format has several advantages: * The CONFIG block need not be moved. * The entire compressed image (depacker and payload) need not be moved another time before the UPX depacker's own operation. * The CONFIG block always lives at 00602h, and the kernel need not be aware whether it was compressed for detecting which CONFIG block to use. * Support for compressed images beyond 64 KiB for free. (The assembly define TEST_FILL_INIT_TEXT can be passed in NASMENV to test this support with 32 KiB of LFSR output.) * A subsequent commit will shorten the stub to 64 bytes, compared to the prior 32 + 45 = 77 bytes, with no loss of features. (The packed payload is a bit shorter too.) * The new stub also sets ds and es to the segment value that would point to the DOS/EXE process's PSP. This is apparently not used by the UPX depacker but could be in a future or past version, or if another packer is used.
The code still allows for up to 192 bytes, should that be useful. Simply by re-assembling upxentry.asm then re-running exeflat, the changes will be picked up.
|
Tested building and running the kernel with Also tested DOS/SYS and forced DOS/EXE compression with my test payload wrapped in kernshim.asm. Note that the kernel command line is not properly passed because testpl expects it in a LOADCMDLINE structure on the stack, differing from the lsvCommandLine buffer used by the FreeDOS load protocol extension, plus testpl does not use the stack address stored by the DOS/EXE entry header stub into the help data. This is an example of the set of commands used to build and pack a test executable: |
Here is an lDebug script I made for checking the LFSR data: https://pushbx.org/ecm/test/20220525.sld (Needs yesterday's or newer lDebug with the LFSR variables.) Using it requires a little setup. Build the kernel with the define enabled: Get the lDebug release, build a boot image with it and the kernel: The entire log of this debugging session is contained in https://pushbx.org/ecm/test/20220526/trace.log Here is just the interesting part: |
|
Finally here is an alternative solution that I explored this week: https://pushbx.org/ecm/test/20220526/fdkernel-big-stub.txt It would require some way to split the compressed image into two parts, which could have been done by |
|
This is looking nice. I haven't had a chance to try yet, just reading through it. |
ecm-pushbx
force-pushed
the
ecm-new-exeflat-format
branch
from
56fa9e1
to
f6681d6
Compare
|
Fixed a comment error in the DOS/SYS device header writer, and there is an error in the PR's message:
Only the DOS/EXE entry is allowed up to 192 bytes. The DOS/SYS entry is allowed up to 176 bytes, as the device header takes up 16 bytes in the layout of the decompressed kernel, and the entry stub plus the header may take up up to 192 bytes. |
|
Added another commit as I noticed the CI build log listed the UPX output first, before the part of the exeflat output that happened before calling UPX. This is a screenshot showing the problem:
Source: https://github.com/FDOS/kernel/runs/6630701821?check_suite_focus=true Appears to be fixed in https://github.com/FDOS/kernel/runs/6630955128?check_suite_focus=true |
Copying the commit message of the first commit in this PR:
This format has several advantages:
The CONFIG block need not be moved.
The entire compressed image (depacker and payload) need not be moved another time before the UPX depacker's own operation.
The CONFIG block always lives at 00602h, and the kernel need not be aware whether it was compressed for detecting which CONFIG block to use.
Support for compressed images beyond 64 KiB for free. (The assembly define TEST_FILL_INIT_TEXT can be passed in NASMENV to test this support with 32 KiB of LFSR output.)
A subsequent commit will shorten the stub to 64 bytes, compared to the prior 32 + 45 = 77 bytes, with no loss of features. (The packed payload is a bit shorter too.) (PR: This commit is included in the PR.)
The new stub also sets ds and es to the segment value that would point to the DOS/EXE process's PSP. This is apparently not used by the UPX depacker but could be in a future or past version, or if another packer is used.
Additional PR comments: There are more commits that add DOS/SYS format compression, as well as the ability to force DOS/SYS or DOS/EXE format compression by omitting the other format's entry file switch to exeflat. The new entry header stub can be assembled into anything between 32 and 192 bytes, exeflat need not be modified to allow longer entry files (up to 192 bytes).