Skip to content

Commit

Permalink
Fixed #18687: Removed test_performance_scalability
Browse files Browse the repository at this point in the history
Even after repeated adjustment of the constants, this test still fails
randomly. It has educated us to ignore messages from Jenkins, to a
point where we missed some actual failures. In addition, it accounts
for a non-negligible percentage of the run time of the test suite
just by itself. Since no one has proposed a convincing patch in months,
I'm going to remove the patch. We can't keep a randomly failing test
forever.
  • Loading branch information
aaugustin committed Sep 6, 2012
1 parent 3f53cac commit 5262a28
Showing 1 changed file with 0 additions and 18 deletions.
18 changes: 0 additions & 18 deletions tests/regressiontests/utils/crypto.py
Expand Up @@ -144,21 +144,3 @@ def test_regression_vectors(self):
result = pbkdf2(**vector['args'])
self.assertEqual(binascii.hexlify(result).decode('ascii'),
vector['result'])

def test_performance_scalability(self):
"""
Theory: If you run with 100 iterations, it should take 100
times as long as running with 1 iteration.
"""
# These values are chosen as a reasonable tradeoff between time
# to run the test suite and false positives caused by imprecise
# measurement.
n1, n2 = 200000, 800000
elapsed = lambda f: timeit.Timer(f,
'from django.utils.crypto import pbkdf2').timeit(number=1)
t1 = elapsed('pbkdf2("password", "salt", iterations=%d)' % n1)
t2 = elapsed('pbkdf2("password", "salt", iterations=%d)' % n2)
measured_scale_exponent = math.log(t2 / t1, n2 / n1)
# This should be less than 1. We allow up to 1.2 so that tests don't
# fail nondeterministically too often.
self.assertLess(measured_scale_exponent, 1.2)

1 comment on commit 5262a28

@jezdez
Copy link
Contributor

@jezdez jezdez commented on 5262a28 Sep 6, 2012

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amen!

Please sign in to comment.