What's new
- Extended masking to address #124 (@Quinntyx in #143)
- Allows PyLingual to decompile long dictionaries, lists, parameters, arguments, etc.
- Optional Redis translation cache to address #153
- New
-r/--redis-host and --redis-port CLI options
- Speeds up the translation step of decompilation
- Added a
docker-compose.yml for easy setup (see the README for details)
- New models for 3.14 and 3.15
- Added missing opnames to the statement tokenizer that led to overconsumption of the token budget
- Control flow reconstruction
- Added templates for 3.15, with downstream improvements to 3.11+ (#165)
- Masking changes
- Fixed number attribute access (
1.to_bytes() → (1).to_bytes())
- Segmentation and equivalence testing
- Transactional segmentation correction to help keep source/CFT consistent (@Henglie in #154)
- 3.13 bytecode patch for the
GET_ITER difference introduced in later 3.13 versions
- Code objects are now hashed by bytecode, name and first line, so identical stub functions don't collide
- Performance
- The original patched
.pyc and unchanged code objects are reused across equivalence checks; the control flow graph is skipped when bytecode matches
- Unmasked lines and rendered sources are cached across segmentation candidates
- Replaced the transitive closure with interval numbering for dominators: O(V²) → O(V)
- Hugging Face is no longer contacted when the model is already downloaded
- GPU memory is freed between files, fixing out-of-memory errors when decompiling many files in a row
uv python find results are reused instead of resolving through uvx on every compile (cherry-picked from @code-reverse #147)
- Use CPU instead of MPS on Apple silicon (cherry-picked from @code-reverse #147)
- Bug fixes
- Fixed silent batching failure in translation (#153)
- Fixed
ValueError: code: co_nlocals != len(co_varnames) (#152)
V1 dataset evaluation
| Version |
v0.0.1 |
v0.1.0 |
delta(Δ) |
| 3.6 |
85.36% |
87.18% |
+1.82% |
| 3.7 |
83.84% |
85.61% |
+1.77% |
| 3.8 |
82.53% |
84.38% |
+1.85% |
| 3.9 |
84.82% |
86.65% |
+1.83% |
| 3.10 |
84.61% |
86.70% |
+2.09% |
| 3.11 |
86.70% |
89.40% |
+2.70% |
| 3.12 |
84.36% |
87.06% |
+2.70% |
| 3.13 |
70.96% |
84.78% |
+13.82% |
| 3.14 |
70.81% |
84.03% |
+13.22% |
| 3.15 |
70.89% |
83.97% |
+13.08% |
V2 dataset evaluation
| Version |
v0.0.1 |
v0.1.0 |
delta(Δ) |
| 3.6 |
15.55% |
26.58% |
+11.03% |
| 3.7 |
13.36% |
23.29% |
+9.93% |
| 3.8 |
9.97% |
19.63% |
+9.66% |
| 3.9 |
16.82% |
26.73% |
+9.91% |
| 3.10 |
20.22% |
31.17% |
+10.95% |
| 3.11 |
13.48% |
29.96% |
+16.48% |
| 3.12 |
15.42% |
29.37% |
+13.95% |
| 3.13 |
40.17% |
61.30% |
+21.13% |
| 3.14 |
1.04% |
38.90% |
+37.86% |
| 3.15 |
10.39% |
42.88% |
+32.49% |
Full Changelog: v0.0.1...v0.1.0
Find the dataset and instructions to reproduce here Pylingual Evaluation Dataset.