Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve DeprecationWarning: Seeding based on hashing #656

Merged
merged 3 commits into from
Mar 12, 2024
Merged

Resolve DeprecationWarning: Seeding based on hashing #656

merged 3 commits into from
Mar 12, 2024

Conversation

bmos
Copy link
Contributor

@bmos bmos commented Mar 11, 2024

When calling randomtable without providing a seed, the warning DeprecationWarning: Seeding based on hashing is deprecated since Python 3.9 and will be removed in a subsequent version. The only supported seed types are: None, int, float, str, bytes, and bytearray. is emitted.

This actually becomes a TypeError in 3.11 and 3.12.

closes #657

It also seems like we don't even need to provide a seed based on the documentation, but I stuck to the way it was implemented before (using a hashed version of the date).
2.7: https://docs.python.org/2.7/library/random.html#random.seed
3.12: https://docs.python.org/3.12/library/random.html#random.seed

I also noticed that the filename utils/random.py seems to be conflicting with python's random module. When debugging the doctests in that file, I had to rename it.

Changes

  • Add unit tests for randomtable, dummytable, and their supporting functions and classes and updated their doctests to pass.
  • Fix: DeprecationWarning: Seeding based on hashing is deprecated since Python 3.9 and will be removed in a subsequent version.

Checklist

Use this checklist to ensure the quality of pull requests that include new code and/or make changes to existing code.

  • Source Code guidelines:
    • Includes unit tests
    • New functions have docstrings with examples that can be run with doctest
    • New functions are included in API docs
    • Docstrings include notes for any changes to API or behavior
    • All changes are documented in docs/changes.rst
  • Versioning and history tracking guidelines:
    • Using atomic commits whenever possible
    • Commits are reversible whenever possible
    • There are no incomplete changes in the pull request
    • There is no accidental garbage added to the source code
  • Testing guidelines:
    • Tested locally using tox / pytest
    • Rebased to master branch and tested before sending the PR
    • Automated testing passes (see CI)
    • Unit test coverage has not decreased (see Coveralls)
  • State of these changes is:
    • Just a proof of concept
    • Work in progress / Further changes needed
    • Ready to review
    • Ready to merge

@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@bmos bmos marked this pull request as ready for review March 11, 2024 02:01
@coveralls
Copy link

coveralls commented Mar 11, 2024

Pull Request Test Coverage Report for Build 8238402266

Details

  • 56 of 58 (96.55%) changed or added relevant lines in 2 files are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage increased (+0.03%) to 91.072%

Changes Missing Coverage Covered Lines Changed/Added Lines %
petl/util/random.py 14 16 87.5%
Files with Coverage Reduction New Missed Lines %
petl/version.py 2 81.82%
Totals Coverage Status
Change from base Build 5940841807: 0.03%
Covered Lines: 13352
Relevant Lines: 14661

💛 - Coveralls

petl/util/random.py Fixed Show fixed Hide fixed
columns, rows = 3, 10
table = randomtable(columns, rows)

assert len(table[0]) == columns

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
columns, rows = 4, 60
table = RandomTable(numflds=columns, numrows=rows, wait=0, seed=None)

assert len(table[0]) == columns

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
fields=columns,
)
assert len(table[0]) == 4
assert len(table) == rows + 1

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
table = randomtable(columns, rows)

assert len(table[0]) == columns
assert len(table) == rows + 1

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
rows = 70
table = dummytable(numrows=rows, fields=None, wait=0, seed=None)

assert len(table) == rows + 1

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
numrows=rows,
seed=42
)
assert len(table[0]) == 3

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
table = RandomTable(numflds=columns, numrows=rows, wait=0, seed=None)

assert len(table[0]) == columns
assert len(table) == rows + 1

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
table = randomtable(columns, rows)

assert len(table[0]) == columns
assert len(table) == rows + 1

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
fields=columns,
)
assert len(table[0]) == 4
assert len(table) == rows + 1

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
rows = 70
table = dummytable(numrows=rows, fields=None, wait=0, seed=None)

assert len(table) == rows + 1

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
numrows=rows,
)
assert len(table[0]) == 3
assert len(table) == rows + 1

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
table = dummytable(
numrows=rows,
)
assert len(table[0]) == 3

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
numrows=rows,
seed=42
)
assert len(table[0]) == 3

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
seed=42
)
assert len(table[0]) == 3
assert len(table) == rows + 1

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
table = RandomTable(numflds=columns, numrows=rows, wait=0, seed=None)

assert len(table[0]) == columns
assert len(table) == rows + 1

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
rows = 35

table = dummytable(numrows=rows, fields=columns)
assert table[0] == ('count', 'pet', 'color', 'value')

Check warning

Code scanning / Bandit (reported by Codacy)

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code. Warning

Use of assert detected. The enclosed code will be removed when compiling to optimised byte code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
rows = 35

table = dummytable(numrows=rows, fields=columns)
assert table[0] == ('count', 'pet', 'color', 'value')

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Fixed Show fixed Hide fixed
petl/test/util/test_random.py Dismissed Show dismissed Hide dismissed
petl/test/util/test_random.py Dismissed Show dismissed Hide dismissed
petl/test/util/test_random.py Dismissed Show dismissed Hide dismissed

assert isinstance(seed_1, str)
assert seed_1 != ""
assert seed_1 != seed_2

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
time.sleep(1)
seed_2 = randomseed()

assert isinstance(seed_1, str)

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.
seed_2 = randomseed()

assert isinstance(seed_1, str)
assert seed_1 != ""

Check notice

Code scanning / Semgrep (reported by Codacy)

The application was found using assert in non-test code. Note

The application was found using assert in non-test code.

# construct fields
flds = ['f%s' % n for n in range(nf)]
flds = ["f%s" % n for n in range(nf)]

Check warning

Code scanning / Prospector (reported by Codacy)

Formatting a regular string which could be a f-string (consider-using-f-string) Warning

Formatting a regular string which could be a f-string (consider-using-f-string)
petl/util/random.py Dismissed Show dismissed Hide dismissed
yield tuple(flds)

# construct data rows
for _ in xrange(nr):
# artificial delay
if self.wait:
time.sleep(self.wait)
yield tuple(random.random() for n in range(nf))
yield tuple(pyrandom.random() for n in range(nf))

Check warning

Code scanning / Bandit (reported by Codacy)

Standard pseudo-random generators are not suitable for security/cryptographic purposes. Warning

Standard pseudo-random generators are not suitable for security/cryptographic purposes.
petl/util/random.py Show resolved Hide resolved
@@ -0,0 +1,93 @@
import random as pyrandom

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Missing module docstring Warning

Missing module docstring
"""
columns = (
('count', partial(pyrandom.randint, 0, 100)),
('pet', partial(pyrandom.choice, ['dog', 'cat', 'cow', ])),

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Module 'random' has no 'choice' member Warning

Module 'random' has no 'choice' member
and that it accepts and uses custom column names provided.
"""
columns = (
('count', partial(pyrandom.randint, 0, 100)),

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Module 'random' has no 'randint' member Warning

Module 'random' has no 'randint' member
columns = (
('count', partial(pyrandom.randint, 0, 100)),
('pet', partial(pyrandom.choice, ['dog', 'cat', 'cow', ])),
('color', partial(pyrandom.choice, ['yellow', 'orange', 'brown'])),

Check warning

Code scanning / Pylintpython3 (reported by Codacy)

Module 'random' has no 'choice' member Warning

Module 'random' has no 'choice' member
petl/util/random.py Dismissed Show dismissed Hide dismissed
fields=(
('foo', partial(pyrandom.randint, 0, 100)),
('bar', partial(pyrandom.choice, ('apples', 'pears', 'bananas', 'oranges'))),
('baz', pyrandom.random),

Check warning

Code scanning / Pylint (reported by Codacy)

Wrong hanging indentation (remove 4 spaces). Warning

Wrong hanging indentation (remove 4 spaces).
numrows=100,
fields=(
('foo', partial(pyrandom.randint, 0, 100)),
('bar', partial(pyrandom.choice, ('apples', 'pears', 'bananas', 'oranges'))),

Check warning

Code scanning / Pylint (reported by Codacy)

Wrong hanging indentation (remove 4 spaces). Warning

Wrong hanging indentation (remove 4 spaces).
and that it accepts and uses custom column names provided.
"""
columns = (
('count', partial(pyrandom.randint, 0, 100)),

Check warning

Code scanning / Pylint (reported by Codacy)

Module 'random' has no 'randint' member Warning

Module 'random' has no 'randint' member
columns = (
('count', partial(pyrandom.randint, 0, 100)),
('pet', partial(pyrandom.choice, ['dog', 'cat', 'cow', ])),
('color', partial(pyrandom.choice, ['yellow', 'orange', 'brown'])),

Check warning

Code scanning / Pylint (reported by Codacy)

Module 'random' has no 'choice' member Warning

Module 'random' has no 'choice' member
"""
columns = (
('count', partial(pyrandom.randint, 0, 100)),
('pet', partial(pyrandom.choice, ['dog', 'cat', 'cow', ])),

Check warning

Code scanning / Pylint (reported by Codacy)

Module 'random' has no 'choice' member Warning

Module 'random' has no 'choice' member
@@ -0,0 +1,93 @@
import random as pyrandom

Check warning

Code scanning / Pylint (reported by Codacy)

Missing module docstring Warning

Missing module docstring
Copy link
Member

@juarezr juarezr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@juarezr juarezr added the Maintainability Issues for code modernization, improve development, testing label Mar 12, 2024
@juarezr juarezr merged commit 140f87b into petl-developers:master Mar 12, 2024
36 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintainability Issues for code modernization, improve development, testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DeprecationWarning / TypeError: Seeding based on hashing is deprecated/removed
4 participants