Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache look-up for functions with optional arguments fails ... #4959

Closed
asodeur opened this issue Dec 13, 2019 · 4 comments
Closed

Cache look-up for functions with optional arguments fails ... #4959

asodeur opened this issue Dec 13, 2019 · 4 comments
Labels
bug caching Issue involving caching

Comments

@asodeur
Copy link
Contributor

asodeur commented Dec 13, 2019

... for non-singleton default arguments (ie booleans and small ints seem ok, floats fail).

from numba import jit

DEFAULT = 123456789

@jit(nopython=True, cache=True)
def x_or_default(x=DEFAULT):
    return x


x_or_default()

# looks like the compile result gets written into a .nbc file
overloads = x_or_default._cache._cache_file._load_index()
print('Cached overloads: ', overloads)

# but you cannot load the compile result from cache
assert x_or_default._cache.load_overload(
        (types.Omitted(DEFAULT),), x_or_default.targetctx
    ) is not None

# the cause seems to be that the unpickled Omitted(DEFAULT) does not equal
# the newly created one, ie he following fails
assert list(overloads)[0][0][0] == types.Omitted(DEFAULT)
assert list(overloads)[0][0][0] == list(
            x_or_default._cache._cache_file._load_index())[0][0][0]

The code above will work for DEFAULT=123 (likely due to Python's small int caching)

@asodeur
Copy link
Contributor Author

asodeur commented Dec 13, 2019

The issue seems to be with the definition of Omitted.key:

@property
def key(self):
   return type(self.value), id(self.value)

To get the above working you had to return type(self.value), self.value. But that will break for any unhashable default value. Maybe need to keep the current behaviour for unhashable default values.

@stuartarchibald
Copy link
Contributor

Thanks for the report. I can reproduce, even the round trip fails:

from numba import jit
from numba import types

DEFAULT = 123456789

@jit(nopython=True, cache=True)
def x_or_default(x=DEFAULT):
    return x


x_or_default()

# looks like the compile result gets written into a .nbc file
overloads = x_or_default._cache._cache_file._load_index()
print('Cached overloads: ', overloads)

# Round trip fails
key = [_ for _ in overloads][0]
assert x_or_default._cache.load_overload(key, x_or_default.targetctx)

@stuartarchibald stuartarchibald added bug caching Issue involving caching and removed needtriage labels Dec 16, 2019
@njriasan
Copy link
Contributor

@stuartarchibald is this issue resolved by #7422? Can this issue be closed?

@stuartarchibald
Copy link
Contributor

@stuartarchibald is this issue resolved by #7422? Can this issue be closed?

@njriasan I think so, OP now works as expected. Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug caching Issue involving caching
Projects
None yet
Development

No branches or pull requests

3 participants