Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
bytecode.rpyb recompiled when there's no changes #1647
While working on RenPyWeb I noted that bytecode.rpyb was updated even when there was no change in any .rpy file.
I tracked the culprit in ast.py:
Ren'Py checks if the Python code located in a .rpyc file was already compiled by checking whether its pre-computed hash is referenced in bytecode.rpyb. This pre-computed hash changes at each compilation, even if the resulting bytecode is the exact same, due to using
Since the .rpyc/.rpymc files are recompiled every time one makes a new Distribution (with Force recompile), this means a game's bytecode.rpyb never carries the same hashes even when compiling the same snippets twice in a row. Consequently the bytecode hashes referenced in the
It seems to me that the location of the Python snippet does not matter as long as the resulting bytecode is bit-for-bit identical. I tested with:
and with this bytecode.rpyb is left untouched, unless there was an actual change in the Python snippet.
I understand that we ship a full renpy/ directory in a Distribution anyway (we'll do that even in renpyweb eventually), so this unnecessary recompilation does not happen often.
Did I miss something?
I think you did miss something.
The main thing is that:
Doesn't run during a "normal" run of Ren'Py, one where no script file has changed. During a normal run, the .rpy file won't change, and hence Ren'Py will load in the .rpyc file instead (see
Because of that, the hash won't change, the bytecode will be looked up, the bytecode cache won't get dirtied, and Ren'Py will start fast.
So I think there's a problem here, but my bet is that something is changing the md5 of the .rpy file between the time it is created and the time it is loaded into Ren'Py, and that's causing it to be recompiled unnecessarily.
(Ren'Py actively rejects Reproducible Builds. The .rpyc files contain annotations needed to make a game loadable - annotations that if included in the script, would make the scripts unbearable to write.)
Note: I'm talking about Python snippets compilation only, not about the .rpy->.rpyc AST-compilation.
I did see that in the common case the bytecode isn't dirtied.
Again, that's a confusing corner-case situation, that could be avoided by using a stable hash for the Python snippets bytecode. If we care to.
(I don't think .rpyc files are a reason to "reject" Reproducible Builds. From what I understand .rpyc are partial source files too. For clarity we could even have 3 files: the source .rpy, the annotations that complement the .rpy (e.g. a ".rpya"), and the purely-derived/compiled parts in a .rpyc that could be safely removed/recreated. Anyway, just an aside.)