ipsw dsc slide --json produces invalid JSON

[AustinTamb]

What happened?

ipsw dsc --json produces invalid JSON. In particular, it will have multiple record (two arrays).

How can we reproduce this?

1. Split DSC:
   ipsw dsc split --json -o /tmp /path/to/cache

2. Validate output with simple python script:

import json

with open("/tmp/slide_info.json", "r") as fd:
    json.load(fd)

This will cause an error to occur:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py", line 293, in load
    return loads(fp.read(),
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/json/decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 22032792)

ipsw version

Version: 3.1.462, BuildTime: 2024-03-24T06:03:31Z

Search

• I did search for other open and closed issues before opening this

Code of Conduct

• I agree to follow this project's Code of Conduct

Additional context

Issue happened when using UniversalMac_14.4_23E214_Restore.ipsw,

[blacktop]

What is the contents of the output file look like?

[AustinTamb]

Problem string:
[  <-- Bad, this is in the middle of the JSON.
    {
        "cache_file_offset": 1692663784,
        "cache_vm_address": 8135114728,
        "target": 8049909977,
        "pointer": {
            "value": 1607459033,
            "next": 1
        }
    },
    {
        "cache_file_offset": 1692663792,
        "cache_vm_address": 8135114736,
        "target": 8049910296,
        "pointer": {
            "value": 1607459352,
            "next": 1
        }
    },
    {
        "cache_file_offset": 1692663800,
        "cache_vm_address": 8135114744,
        "target": 8049909983,
        "pointer": {
            "value": 1607459039,
            "next": 0
        }
    }
] <-- Bad
[ <-- Bad
    {
        "cache_file_offset": 1692680296,
        "cache_vm_address": 8168685672,
        "target": 6443202716,
        "pointer": {
            "value": 751772,
            "next": 2
        }
    },
    {
        "cache_file_offset": 1692680312,
        "cache_vm_address": 8168685688,
        "target": 8267500200,
        "pointer": {
            "value": 1825049256,
            "next": 2
        },
        "symbol": "__OBJC_$_PROTOCOL_INSTANCE_METHODS_NSObject"
    },
    {
        "cache_file_offset": 1692680328,
        "cache_vm_address": 8168685704,
        "target": 8267500664,
        "pointer": {
            "value": 1825049720,
            "next": 2
        },
        "symbol": "__OBJC_$_PROTOCOL_INSTANCE_METHODS_OPT_NSObject"
    },
    {
        "cache_file_offset": 1692680344,
        "cache_vm_address": 8168685720,
        "target": 8267500696,
        "pointer": {
            "value": 1825049752,
            "next": 2
        },
        "symbol": "__OBJC_$_PROP_LIST_NSObject"
    }

[blacktop]

So it's multiline JSON, where the ] [ should be a boundary of the JSONL line.

This is because each line goes with a different DSC mapping. I usually filter the results w/ jq which can handle multi-list JSON just fine.

So you'd either include a preprocessing step to break it into several JSON files OR I could create multiple JSON files when the --output flag is used in ipsw

[AustinTamb]

I mean, knowing it's JSONL makes this much less of a problem. I think maybe updating the help message to specifying the output is in JSONL format would probably be sufficient.

[blacktop]

I pushed a commit that now names the file .jsonl to notify people that it's not normal JSON

[blacktop]

is in latest release