RFE: Option to hardlink duplicate bytecache files of different opt. levels

[hroncok]

I'd like to have and option of compileall that deduplicates identical .pyc files of the same module if compiled for different optimization levels.

Example:

$ python -m compileall -o0 -o1 -o2 --hardlink-dupes ...

This would hardlink module.cpython-3?.pyc with module.cpython-3?.opt-1.pyc with module.cpython-3?.opt-1.pyc if identical, on operating systems supporting hardlinks.

Given the nature of the bytecode caches, the non-optimized, optimized level 1 and optimized level 2 .pyc files may or may not be identical.

Consider the following Python module:

1

All three bytecode cache files would by identical.

While with:

assert 1

Only the two optimized cache files would be identical with each other.

And this:

"""Dummy module docstring"""
1

Would produce two identical bytecode cache files but the opt-2 file would differ.

Only modules like this would produce 3 different files:

"""Dummy module docstring"""
assert 1

Hardlinking identical .pyc files can cause significant storage savings. As a single data point: On my workstation I have 360 MiB of various Python 3.7 bytecode files in /usr and I can save 108 MiB.

[hroncok]

https://discuss.python.org/t/compileall-option-to-hardlink-duplicate-optimization-levels-bytecode-cache-files/3014