Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve section count guessing #3

Merged
merged 1 commit into from
Apr 9, 2023
Merged

Improve section count guessing #3

merged 1 commit into from
Apr 9, 2023

Conversation

AT0myks
Copy link
Contributor

@AT0myks AT0myks commented Apr 8, 2023 •
edited
Loading

I was running some tests on 416 different PAK files and noticed that guess_section_count does not give a correct result for some of them.
4/416 are RLN36 firmwares that we can ignore for now, and out of the other 412 the guess is wrong for 25 of them.

This is an example of the result before:

pakler -l IPC_5158MP8M.17_20042201.D800.IMX317.8MP.REOLINK.pak
Attempting to guess number of sections... guessed: 8
Header  magic=32725913  crc32=5a5a9a01  type=00002602  sections=<8>  mtd_parts=<8>
    Section  0 name="loader"         version="v1.0.0.1"       start=00000610  len=00008000  (start=    1552 len=   32768)
    Section  1 name="ext"            version="v1.0.0.1"       start=00008610  len=00000a28  (start=   34320 len=    2600)
    Section  2 name="uitron"         version="v1.0.0.1"       start=00009038  len=000d624c  (start=   36920 len=  877132)
    Section  3 name="uboot"          version="v1.0.0.1"       start=000df284  len=0004073c  (start=  914052 len=  263996)
    Section  4 name=""               version=""               start=0011f9c0  len=00000000  (start= 1178048 len=       0)
    Section  5 name="kernel"         version="v1.0.0.1"       start=0011f9c0  len=0017d882  (start= 1178048 len= 1562754)
    Section  6 name="fs"             version="v1.0.0.1"       start=0029d242  len=0028a000  (start= 2740802 len= 2662400)
    Section  7 name=""               version=""               start=00527242  len=00000000  (start= 5403202 len=       0)
    Mtd_part name=""               mtd="BrR"  a=00000000  start=00000000  len=00000000
    Mtd_part name=""               mtd="BrR"     a=00000000  start=00000000  len=00000000
    Mtd_part name=""               mtd="loader"      a=00527242  start=00000000  len=00000000
    Mtd_part name="/dev/mtd9"      mtd="☺ext"         a=00000000  start=00000000  len=00010000
    Mtd_part name="/dev/mtd9"      mtd="☺uitron"      a=00010000  start=00000000  len=00020000
    Mtd_part name="/dev/mtd9"      mtd="¶uboot"       a=00020000  start=00000000  len=00160000
    Mtd_part name="/dev/mtd9"      mtd="♣bootargs"    a=00160000  start=00000000  len=ffffffff
    Mtd_part name="/dev/mtd9"      mtd="kernel"      a=ffffffff  start=00000000  len=001b0000
CRC MISMATCH, file: IPC_5158MP8M.17_20042201.D800.IMX317.8MP.REOLINK.pak, header.crc=5a5a9a01, got=600514a4

and now:

pakler -l IPC_5158MP8M.17_20042201.D800.IMX317.8MP.REOLINK.pak
Attempting to guess number of sections... guessed: 11
Header  magic=32725913  crc32=5a5a9a01  type=00002602  sections=<11>  mtd_parts=<11>
    Section  0 name="loader"         version="v1.0.0.1"       start=00000610  len=00008000  (start=    1552 len=   32768)
    Section  1 name="ext"            version="v1.0.0.1"       start=00008610  len=00000a28  (start=   34320 len=    2600)
    Section  2 name="uitron"         version="v1.0.0.1"       start=00009038  len=000d624c  (start=   36920 len=  877132)
    Section  3 name="uboot"          version="v1.0.0.1"       start=000df284  len=0004073c  (start=  914052 len=  263996)
    Section  4 name=""               version=""               start=0011f9c0  len=00000000  (start= 1178048 len=       0)
    Section  5 name="kernel"         version="v1.0.0.1"       start=0011f9c0  len=0017d882  (start= 1178048 len= 1562754)
    Section  6 name="fs"             version="v1.0.0.1"       start=0029d242  len=0028a000  (start= 2740802 len= 2662400)
    Section  7 name=""               version=""               start=00527242  len=00000000  (start= 5403202 len=       0)
    Section  8 name=""               version=""               start=00527242  len=00000000  (start= 5403202 len=       0)
    Section  9 name=""               version=""               start=00527242  len=00000000  (start= 5403202 len=       0)
    Section 10 name=""               version=""               start=00527242  len=00000000  (start= 5403202 len=       0)
    Mtd_part name="loader"         mtd="/dev/mtd9"       a=00000000  start=00000000  len=00010000
    Mtd_part name="ext"            mtd="/dev/mtd9"       a=00010000  start=00010000  len=00010000
    Mtd_part name="uitron"         mtd="/dev/mtd9"       a=00020000  start=00020000  len=00140000
    Mtd_part name="uboot"          mtd="/dev/mtd9"       a=00160000  start=00160000  len=00050000
    Mtd_part name="bootargs"       mtd="/dev/mtd9"       a=ffffffff  start=ffffffff  len=00000000
    Mtd_part name="kernel"         mtd="/dev/mtd9"       a=001b0000  start=001b0000  len=001c0000
    Mtd_part name="fs"             mtd="/dev/mtd9"       a=00370000  start=00370000  len=00400000
    Mtd_part name="app"            mtd="/dev/mtd9"       a=ffffffff  start=ffffffff  len=00000000
    Mtd_part name="para"           mtd="/dev/mtd9"       a=00770000  start=00770000  len=00080000
    Mtd_part name="NOUSE"          mtd="/dev/mtd9"       a=ffffffff  start=ffffffff  len=00000000
    Mtd_part name="ptzmcu"         mtd="/dev/mtd9"       a=ffffffff  start=ffffffff  len=00000000
File passes CRC check: IPC_5158MP8M.17_20042201.D800.IMX317.8MP.REOLINK.pak

The code is based on a few assumptions:

  • no 2 sections have the same name
  • first section's name is the same as the first MTD's name
  • there can't be more than 30 sections

It simply keeps reading SECTION_SIZE bytes until the result starts with the name of the first section, meaning we have reached the MTDs.

Also, I haven't seen a single PAK file that doesn't follow these "rules".

The "new" sections for these 25 firmwares are all empty, so this is more of a fix for the MTDs.
Nothing changes for the other 387 firmwares.

@vmallet
Copy link
Owner

vmallet commented Apr 9, 2023

Very nice, thank you! Would you mind adding a bit of the very nice explanation you put here (or all of it) directly into the commit message? It would be good to have that tied to the code when things get forked and such.

@AT0myks
Improve section count guessing
54ea25a 
This change is based on a few assumptions:
- no 2 sections have the same name
- first section's name is the same as the first MTD's name
- there can't be more than 30 sections
We keep reading SECTION_SIZE bytes until the result starts with
the name of the first section, meaning we have reached the MTDs.
As of now, all PAK files seem to follow these rules.
@AT0myks
Copy link
Contributor Author

AT0myks commented Apr 9, 2023

Done, I hope I didn't break anything.

@vmallet vmallet merged commit 0722511 into vmallet:main Apr 9, 2023
@AT0myks AT0myks deleted the section-count branch April 10, 2023 05:13
@AT0myks AT0myks mentioned this pull request Apr 10, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@AT0myks @vmallet