Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-110932: Fix regrtest for SOURCE_DATE_EPOCH #111143

Merged
merged 1 commit into from Oct 21, 2023
Merged

gh-110932: Fix regrtest for SOURCE_DATE_EPOCH #111143

merged 1 commit into from Oct 21, 2023

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Oct 20, 2023 •
edited by bedevere-app bot

If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.

@vstinner
pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH
55bf9f7 
If the SOURCE_DATE_EPOCH environment variable is defined, use its
value as the random seed.
@bedevere-app bedevere-app bot mentioned this pull request Oct 20, 2023
Closed
@vstinner
Copy link
Member Author

vstinner commented Oct 20, 2023 •
edited

SOURCE_DATE_EPOCH intent is to make programs reproducible. Without this change, if SOURCE_DATE_EPOCH is set, regrtest uses a random seed which makes tests less reproducible.

With this change, the random seed is constant and so the behavior should be more reproducible.

--randseed=custom_string command line option is invalid: the option value must be an integer. Only SOURCE_DATE_EPOCH is used to use a string as the random seed.

@vstinner
Copy link
Member Author

@sobolevn: Would you mind to review this change?

sobolevn
sobolevn reviewed Oct 21, 2023
View reviewed changes
Copy link
Member

@sobolevn sobolevn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks good to me, just one idea.

Lib/test/libregrtest/main.py
else random.getrandbits(32)
)
if 'SOURCE_DATE_EPOCH' in os.environ:
if ('SOURCE_DATE_EPOCH' in os.environ
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

Suggested change
if ('SOURCE_DATE_EPOCH' in os.environ
if (
'SOURCE_DATE_EPOCH' in os.environ

Lib/test/libregrtest/main.py
# SOURCE_DATE_EPOCH should be an integer, but use a string to not
# fail if it's not integer. random.seed() accepts a string.
# https://reproducible-builds.org/docs/source-date-epoch/
self.random_seed: int | str = os.environ['SOURCE_DATE_EPOCH']
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Myabe we should always use str? There's no real reason to use int here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason is subtle. It avoids to import hashlib to convert a string to an integer in Random.seed(). Right now, hashlib is always imported, but I have a local branch to reduce the number of imports at Python startup.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm talking about the second code path, where we create a random seed as an integer.

Lib/test/libregrtest/main.py
# https://reproducible-builds.org/docs/source-date-epoch/
self.random_seed: int | str = os.environ['SOURCE_DATE_EPOCH']
elif ns.random_seed is None:
self.random_seed = random.getrandbits(32)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If so, it would become:

Suggested change
self.random_seed = random.getrandbits(32)
self.random_seed = str(random.getrandbits(32))

@vstinner vstinner merged commit 7237fb5 into python:main Oct 21, 2023
25 checks passed
@vstinner vstinner deleted the regrtest_epoch branch October 21, 2023 08:37
@vstinner vstinner added needs backport to 3.11 only security fixes needs backport to 3.12 bug and security fixes labels Oct 21, 2023
@miss-islington-app
Copy link

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

@miss-islington-app
Copy link

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 21, 2023
@vstinner @miss-islington
pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH (pythonGH-111143)
76f4d89 
If the SOURCE_DATE_EPOCH environment variable is defined, use its
value as the random seed.
(cherry picked from commit 7237fb5)

Co-authored-by: Victor Stinner <vstinner@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 21, 2023
@vstinner @miss-islington
pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH (pythonGH-111143)
91ad185 
If the SOURCE_DATE_EPOCH environment variable is defined, use its
value as the random seed.
(cherry picked from commit 7237fb5)

Co-authored-by: Victor Stinner <vstinner@python.org>
@bedevere-app
Copy link

bedevere-app bot commented Oct 21, 2023

GH-111153 is a backport of this pull request to the 3.11 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.11 only security fixes label Oct 21, 2023
@bedevere-app
Copy link

bedevere-app bot commented Oct 21, 2023

GH-111154 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Oct 21, 2023
@vstinner
Copy link
Member Author

Merged, thanks to reviewing the change @sobolevn.

vstinner added a commit that referenced this pull request Oct 21, 2023
@miss-islington @vstinner
[3.11] gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (GH-111143) (#11…
47670fb 
…1153)

gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (GH-111143)

If the SOURCE_DATE_EPOCH environment variable is defined, use its
value as the random seed.
(cherry picked from commit 7237fb5)

Co-authored-by: Victor Stinner <vstinner@python.org>
vstinner added a commit that referenced this pull request Oct 21, 2023
@miss-islington @vstinner
[3.12] gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (GH-111143) (#11…
0409057 
…1154)

gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (GH-111143)

If the SOURCE_DATE_EPOCH environment variable is defined, use its
value as the random seed.
(cherry picked from commit 7237fb5)

Co-authored-by: Victor Stinner <vstinner@python.org>
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
@vstinner @aisk
pythongh-110932: Fix regrtest for SOURCE_DATE_EPOCH (python#111143)
9017ebb 
If the SOURCE_DATE_EPOCH environment variable is defined, use its
value as the random seed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sobolevn sobolevn sobolevn left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@vstinner @sobolevn