Refactored report format to use Pydantic

[garrett-zetier]

Resolves #1250

Overview

These changes update the JSON report format (--report option) so that pydantic models are used to serialize report data to JSON format rather than attrs. This allows for easier deserialization in the future, and a Pydantic TypeAdapter is provided to make reading from the JSON files back into Python objects easier. A test has been added to test deserialization.

Testing

Update the environment to the latest since pydantic has been added as a dependency.

The tests have been updated to verify the latest report format, so they can be run to verify these changes. Specifically, the test_models.py (run uv run pytest tests/test_models.py -v) verifies both serialization and deserialization.

You can run unblob with the --report option to manually verify the JSON report. For example, run uv run unblob --report report.json my.bin

[qkaiser]

first quick pass over the code, some of my colleagues will also have a look :)

[qkaiser]

Code looks ok, let's run the pipeline to see what fails.

[qkaiser]

Don't forget to run pre-commit before pushing code:

pre-commit install
pre-commit run -a

You can also check typing by running pyright from the repo root.

[vlaci]

I went over the changes and I am mostly happy with them, good job!

The base64 process output encoding is cool, as it is more portable than python's string escapes.

I disliked that we need to fight with the type checker about the discriminator values introducing a bit convoluted inheritance hierarchy. My suggestion would be to use callable discriminators eliminating the need for the report_type field on the models themselves.

I've played around with the idea on how it would look like here: garrett-zetier/unblob@1250-pydantic-report...onekey-sec:unblob:pull/1250/review

[garrett-zetier]

I like your implementation of the report models much better. It is much cleaner. Let me know if you want me to continue with those changes on my fork.

One other thing that seems to be missing is that the CI tests are failing because the pydantic package is missing from some list of dependencies. I'm not sure where this needs to be added (package.nix?).

[vlaci]

I like your implementation of the report models much better. It is much cleaner. Let me know if you want me to continue with those changes on my fork.

If you are up to it, then please do so. If you say that you've done enough, that's fine as well, we can take over.

One other thing that seems to be missing is that the CI tests are failing because the pydantic package is missing from some list of dependencies. I'm not sure where this needs to be added (package.nix?).

Yes, it is missing from the dependencies list in package.nix.

[garrett-zetier]

@vlaci I'll take a look and finish it up. Thank you guys for assisting!

[garrett-zetier]

I believe everything should be addressed. All tests and pipelines should pass now.

[vlaci]

I am happy about the current state of the code

[garrett-zetier]

Thanks everyone for their assistance and time reviewing! I am looking forward to these changes in a future release.

[garrett-zetier]

I have rebased from the latest on the upstream main branch. Is there anywhere I should document usage of the updated report with Pydantic?

[vlaci]

I have rebased from the latest on the upstream main branch. Is there anywhere I should document usage of the updated report with Pydantic?

We don't publish API docs, and this change is API-only, so I don't think that this change needs extra documentation