numpy.testing.assert_equal is highly inefficient for many equality tests

[machow]

Looking at the source, it doesn't seem that this function is really geared toward string comparisons. For example, the code below runs at very different speeds...

records = ['abc']*10000
records2 = ['abc']*10000

# compare
for ii in range(len(records)): records[ii] == records2[ii]

# vs
import numpy as np
np.testing.assert_equal(records, records2)
# or
for ii in range(len(records)): np.testing.assert_equal(records[ii], records2[ii])

Related to #189

[filipsch]

benchmark.py (I know it's ugly)

import random
import time
import numpy as np
import string
import random

class Timer():
 
    def __init__(self, desc):
        self.start = time.time()
        self.desc = desc
 
    def __enter__(self):
        return self
 
    def __exit__(self, exc_type, exc_val, exc_tb):
        end = time.time()
        runtime = end - self.start
        msg = '{desc} took {time} seconds to complete'
        print(msg.format(desc=self.desc, time=round(runtime,5)))
 
if __name__ == '__main__':
    n = 10000
    records = ['abc']*n
    records2 = ['abc']*n
    with Timer('np.testing.assert_equal() for lists'):
        np.testing.assert_equal(records, records2)
    with Timer('np.array_equal() for lists'):
        np.array_equal(records, records2)
    
    print('\n')
    random.seed(123)
    keys = set([ random.choice(string.ascii_letters) + random.choice(string.ascii_letters) + random.choice(string.ascii_letters) + random.choice(string.ascii_letters) for i in range(n)])
    values = [1]*len(keys)
    records3 = dict(zip(keys, values))
    records4 = dict(zip(keys, values))
    with Timer('np.testing.assert_equal() for dicts'):
        np.testing.assert_equal(records3, records4)
    with Timer('np.array_equal() for dicts'):
        np.array_equal(records3, records4)
    
    print('\n')
    records5 = np.array([0.1234]*n)
    records6 = np.array([0.1234]*n)
    with Timer('np.testing.assert_equal() for np arrays'):
        np.testing.assert_equal(records5, records6)
    with Timer('np.array_equal() for np arrays'):
        np.array_equal(records5, records6)

Results:

$ python benchmark.py
python benchmark.py
np.testing.assert_equal() for lists took 0.59289 seconds to complete
np.array_equal() for lists took 0.00117 seconds to complete


np.testing.assert_equal() for dicts took 0.71972 seconds to complete
np.array_equal() for dicts took 0.00082 seconds to complete


np.testing.assert_equal() for np arrays took 0.00042 seconds to complete
np.array_equal() for np arrays took 3e-05 seconds to complete

Takeaways:

• np.testing.assert_equal() is hopelessly slow. np.array_equal() is several orders of magnitudes faster
• np.array_equal() seems to work for lists, dicts and ndarrays.

Todo:

• Add a couple of extra tests to make sure that np.array_equal() shows the required behavior.
• Go ahead and change them out.

This will drastically improve user experience.

[machow]

@filipsch have you seen the timeit library? It's super handy--especially the ipython %%timeit magic command!

https://docs.python.org/3/library/timeit.html

[filipsch]

@machow yes, have seen it, I just cobbled something together.

[machow]

Ah, okay--I thought you had, but wanted to check, since %%timeit changed my life :o. Glad you're working on this issue--kind of crazy how big of an effect it will have across python courses...