Skip to content

v0.1.0

Latest

Choose a tag to compare

@Joel-Flores123 Joel-Flores123 released this 20 Oct 00:23

What's new

  • Extended masking to address #124 (@Quinntyx in #143)
    • Allows PyLingual to decompile long dictionaries, lists, parameters, arguments, etc.
  • Optional Redis translation cache to address #153
    • New -r/--redis-host and --redis-port CLI options
    • Speeds up the translation step of decompilation
    • Added a docker-compose.yml for easy setup (see the README for details)
  • New models for 3.14 and 3.15
    • Added missing opnames to the statement tokenizer that led to overconsumption of the token budget
  • Control flow reconstruction
    • Added templates for 3.15, with downstream improvements to 3.11+ (#165)
  • Masking changes
    • Fixed number attribute access (1.to_bytes() → (1).to_bytes())
  • Segmentation and equivalence testing
    • Transactional segmentation correction to help keep source/CFT consistent (@Henglie in #154)
    • 3.13 bytecode patch for the GET_ITER difference introduced in later 3.13 versions
    • Code objects are now hashed by bytecode, name and first line, so identical stub functions don't collide
  • Performance
    • The original patched .pyc and unchanged code objects are reused across equivalence checks; the control flow graph is skipped when bytecode matches
    • Unmasked lines and rendered sources are cached across segmentation candidates
    • Replaced the transitive closure with interval numbering for dominators: O(V²) → O(V)
    • Hugging Face is no longer contacted when the model is already downloaded
    • GPU memory is freed between files, fixing out-of-memory errors when decompiling many files in a row
    • uv python find results are reused instead of resolving through uvx on every compile (cherry-picked from @code-reverse #147)
    • Use CPU instead of MPS on Apple silicon (cherry-picked from @code-reverse #147)
  • Bug fixes
    • Fixed silent batching failure in translation (#153)
    • Fixed ValueError: code: co_nlocals != len(co_varnames) (#152)

V1 dataset evaluation

Version v0.0.1 v0.1.0 delta(Δ)
3.6 85.36% 87.18% +1.82%
3.7 83.84% 85.61% +1.77%
3.8 82.53% 84.38% +1.85%
3.9 84.82% 86.65% +1.83%
3.10 84.61% 86.70% +2.09%
3.11 86.70% 89.40% +2.70%
3.12 84.36% 87.06% +2.70%
3.13 70.96% 84.78% +13.82%
3.14 70.81% 84.03% +13.22%
3.15 70.89% 83.97% +13.08%

V2 dataset evaluation

Version v0.0.1 v0.1.0 delta(Δ)
3.6 15.55% 26.58% +11.03%
3.7 13.36% 23.29% +9.93%
3.8 9.97% 19.63% +9.66%
3.9 16.82% 26.73% +9.91%
3.10 20.22% 31.17% +10.95%
3.11 13.48% 29.96% +16.48%
3.12 15.42% 29.37% +13.95%
3.13 40.17% 61.30% +21.13%
3.14 1.04% 38.90% +37.86%
3.15 10.39% 42.88% +32.49%

Full Changelog: v0.0.1...v0.1.0
Find the dataset and instructions to reproduce here Pylingual Evaluation Dataset.