Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deterministic code generation #1417

Merged
merged 4 commits into from
May 9, 2022
Merged

Deterministic code generation #1417

merged 4 commits into from
May 9, 2022

Conversation

mstimberg
Copy link
Member

Two main issues led to code files changing for identical models (i.e., even when running a model again in a new process where the problems with regard to names as discussed in #680 do not play a role):

  1. Variable declarations and other "unordered" code was generated in a different order for every run, and
  2. The file names written to the result directory used hash() to deal with the problem of case-insensitivity on Windows (otherwise i and I would be stored to the same file). For security reasons, the hash() function changes its output for every run.

The fix in this PR is to:

  1. Use sorted() whenever iterating over variables that are stored in sets, and
  2. Use zlib.crc32 instead of hash to calculate the hash value for the filenames. Strictly speaking, crc32 is a checksum and not a hash value and is more prone to lead to collisions, but this should not matter for us since we only need to disambiguate in very specific circumstances (names that are identical except for case).

I did not set up a formal test for this, but I verified with fairly complex examples (e.g. frompapers/Kremer_et_al_2011_barrel_cortex.py) that the generated is identical over runs.

Copy link
Member

@thesamovar thesamovar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

@mstimberg mstimberg merged commit 33dfd04 into master May 9, 2022
@mstimberg mstimberg deleted the deterministic_code branch May 9, 2022 08:44
denisalevi added a commit to brian-team/brian2cuda that referenced this pull request May 25, 2022
See brian-team/brian2#1417, should be reverted once Brian2 submodule
tracks that version
denisalevi added a commit to brian-team/brian2cuda that referenced this pull request May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants