Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance when printing large sample sets #1068

Closed
arcondello opened this issue Jan 5, 2022 · 1 comment · Fixed by #1075
Closed

Improve performance when printing large sample sets #1068

arcondello opened this issue Jan 5, 2022 · 1 comment · Fixed by #1075

Comments

@arcondello
Copy link
Member

Currently the dimod.serialization.format.Formatter has poor performance when working with large sample sets.
Some obvious low hanging fruit is to exit early when the depth is exceeded

if depth is None:
but I imagine there are many other improvements.

@arcondello
Copy link
Member Author

Another data point

import time

import dimod
import numpy as np

sampleset = dimod.SampleSet.from_samples(np.ones((91, 500000)), vartype='BINARY', energy=np.zeros(91))


def filter(sampleset, n=100):
    if len(sampleset.variables) > n:
        sampleset = dimod.keep_variables(sampleset, list(sampleset.variables)[:n])
    return sampleset

# no filter
t = time.perf_counter()
dimod.serialization.format.Formatter().format(sampleset)
print(time.perf_counter() - t)

# filter
t = time.perf_counter()
dimod.serialization.format.Formatter().format(filter(sampleset))
print(time.perf_counter() - t)

gives

21.41042103699965
0.05259135400046944

See also #1069 for the weird slicing syntax in the example.

arcondello added a commit to arcondello/dimod that referenced this issue Jan 12, 2022
Note that there are more invasive changes that could
improve this even more - this just adds a sanity check
at the beginning.

Closes dwavesystems#1068
arcondello added a commit to arcondello/dimod that referenced this issue Jan 13, 2022
Note that there are more invasive changes that could
improve this even more - this just adds a sanity check
at the beginning.

Closes dwavesystems#1068
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant