KeyError/PicklingError: Can't pickle class.. it's not found as numba.experimental.jitclass #6032
Comments
|
Debug client attached. During handling of the above exception, another exception occurred: Traceback (most recent call last): Error: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): |
|
@rpopovici thanks for asking about this on the Numba issue tracker. For the time being |
|
@esc can you suggest an alternative solution to this problem? thanks |
|
@rpopovici not sure what the intent is here. Perhaps you could supply a fully runnable reproducer, i.e. something that includes a |
|
@esc I updated the initial post. It should be runnable now.. |
|
@esc by commenting the clone() method in my example, I discovered that calling the class constructor from class method causes the pickle error. I thought that calling the class constructor from class method was supported by numba. Is that a correct assumption or I am mistaken? Can you pls suggest an alternative solution. thanks Also the jupyter notebook kernel crashes for some reason when I try to use nb.deferred_type().. |
|
@rpopovici thanks for following up on this. I'm afraid I don't quite follow what you mean with ", I discovered that calling the class constructor from class method causes the pickle error." I don't see any |
|
@rpopovici also, I am confused about the |
|
@esc sorry for not being very explicit. The clone method it's not used in the example code I pasted here, but it's used in my dev project.. There is no @classmethod, it's the instance method clone(). Just simply by removing any constructor being called in methods proved to make the pickle error go away, but I need to be able to instantiate new objects from methods.. |
|
OK, that is interesting, however in the above example if I comment out the |
|
@esc I simplified the code a little bit. Now there is only one clone method with one constructor call. If you remove this method, then it will work and you can call load_mesh(mesh) without errors |
|
@rpopovici thanks for following up on this. I was assuming, that this error arises from trying to instantiate new object from methods. However, it seems I was mistaken. I created a small reproducer to investigate this, and it seems to work fine: So the question is, what is it about your snippet that causes the error in the first place? I think it would be good to reduce the example to something smaller with less boilerplate to expose the original issue. I am also re-opening this, since you are not trying to pickle a jitclass explicitly and the error you are seeing seems to be a symptom of something else, which may be worth investigating. |
|
@esc thanks. I am looking into it |
|
Incidentally, when taking a closer look at the error message when I run your example locally, I see: So this may in fact be related to the typed list in there. |
|
@esc the culprit appear to be this line: It's unable to deal with the custom list type definition |
|
@rpopovici excellent, so the hypothesis is that using a |
|
@esc I think the problem starts with the fact that it cannot handle numba.typed.List(class_A_type) in class_B definition, hence the KeyError and the pickeling is another issue probably related to the KeyError.. |
|
@rpopovici it might make sense to consult the following section in the docs about this: https://numba.pydata.org/numba-doc/latest/user/jitclass.html#specifying-numba-typed-containers-as-class-members |
|
This is the KeyError I got. I thought I put it in the initial error log, but I didn't.. KeyError Traceback (most recent call last) KeyError: ListType[instance.jitclass.Vertex#26e9c666b48>] During handling of the above exception, another exception occurred: PicklingError Traceback (most recent call last) |
|
yeah, it looks like the type contains an instance of a Vertex, not the type spec, (i think). |
|
yes, I tried to preinitialize the list type with list_type= types.ListType(VertexType), I also tried typeof. None of those work. |
|
and the problem arises only in the class definition with @jitclass.. anywhere else in the code, you can use custom types in typed list |
rpopovici
changed the title
|
Reproducer: from numba import typed, typeof, njit, int64
from numba.experimental import jitclass
spec = [
('value', int64),
]
@jitclass(spec)
class Bag(object):
def __init__(self, value):
self.value = value
def clone(self):
return Bag(self.value)
mybag = Bag(21)
x = typed.List.empty_list(Bag.class_type.instance_type)
x.append(mybag)
@njit
def foo(z):
return z
foo(x)the issue is that |
|
@stuartarchibald is there any workaround to this issue? |
|
This help? from numba import typed, typeof, njit, int64, types
from numba.experimental import jitclass
from numba.extending import overload_method, overload
spec = [
('value', int64),
]
@jitclass(spec)
class Bag(object):
def __init__(self, value):
self.value = value
mybag = Bag(21)
x = typed.List.empty_list(Bag.class_type.instance_type)
x.append(mybag)
@overload_method(types.misc.ClassInstanceType, 'clone')
def ol_bag_clone(inst,):
if inst is Bag.class_type.instance_type:
def impl(inst,):
return Bag(inst.value)
return impl
@njit
def foo(z):
t = z[0]
t.clone()
foo(x)also think the original post needs : @jit(nopython=False)
def put_back(p):
return p[0].vertices[0].pos.x
|
|
@stuartarchibald this definitely heps. thanks! |
Hello guys,
I am trying to use numba with @jitclass. I have multiple classes with class attribute members and typed lists
Do you have any idea why I get this error?
This is a sample code..
And I am using it from this code:
the mesh object is a plain python list of dicts with floats inside..
The text was updated successfully, but these errors were encountered: