Decoding KNeighboursClassifier model from Sklearn fails #192
Comments
|
Thanks for the report, much appreciated. What is Mentioning @EelcoHoogendoorn and @marscher since they're much more familiar with the numpy extension, and probably a lot smarter than me in all things numpy too ;-) Instead of pasting the problem code into this issue, can you please add your testcase to the With the given example above, the calling sequence might have a buglet. I'll try to walk through it here below. This first part looks fairly boilerplate. We could use the top-level jsonpickle.loads/dumps functions but this is using the lower-level API with picklers and unpicklers instead: At this point we have a pickler and unpickler. Next we create a json string The following line looks like a buglet: The buglet above is we're using the The line above is now undoing the double json encoding above. Technically that should work, but it is kinda confusing. Did you truly intend to double-json encode the result from jsonpickle? That said, this should work because the double-encoded json string gets restored by the call to I suspect that we could simplify that part of the test case down to something simpler like this below: I only mention this because it doesn't look like the raw dict is used in a way that needs explicit picklers and unpicklers, but perhaps they're used by other parts of the code that's not relevant to this issue. Let me know about getting a complete test case (regarding the unknown The best part about adding the test case to |
|
hi, which line is actually failing (please show us the stack trace). This error message is popping up, if there is an unpickle-able type involved. By not-pickleable I mean, there is no get/setstate, reduce etc. protocol plus it is not a primitive type. Since you have registered the numpy handlers, this should actually work. When I used jsonpickle, I have written my own handler, so I can not really tell how good it is (eg. does it support exotic dtypes, like structured arrays etc.?). Are you familiar with pdb? You can wrap your script in an try: .... except block and within except call: pdb.post_mortem() to gain insight which object is causing this TypeError. |
|
Here's the stacktrace |
|
I just hit that bug. I was able to narrow it down where it crashed. 100% reproducible. |
|
I am facing same issue while doing something like this
in decoding @davvid please help |
|
I can reproduce import jsonpickle
from enum import Enum
class WrapperClass(object):
class myenum(Enum):
Hello = 1
mydict = {
WrapperClass.myenum.Hello: 'test'
}
print(mydict)
print(jsonpickle.dumps(mydict, keys=True))
print(jsonpickle.loads(jsonpickle.dumps(mydict, keys=True), keys=True))I don't know if this is the same issue @mohitbadwal is noticing with numpy. |
|
Hi, I am also facing same issue, interestingly if I switch to jsonpickle=0.7.2 everything works fine. |
|
There seem to be at least two separate issues in this thread (and both might be different from the OP's issue...). Here's another test case that shows the root of the first problem: import jsonpickle
class Outer(object):
class Inner(object):
pass
print(jsonpickle.encode(Outer.Inner))The output is diff --git a/jsonpickle/util.py b/jsonpickle/util.py
index 8566450..dff7dac 100644
--- a/jsonpickle/util.py
+++ b/jsonpickle/util.py
@@ -507,7 +507,7 @@ def importable_name(cls):
True
"""
- name = cls.__name__
+ name = cls.__qualname__
module = translate_module_name(cls.__module__)
return '%s.%s' % (module, name)
However, this does not fix the decoding, because the decoding code assumes that only the last part of the "importable name" is a class name, and the rest is a module; in this case, it would therefore try to import the It might be cleaner to store the module and class information separately in a tuple, but the following change makes it simply try all options: diff --git a/jsonpickle/unpickler.py b/jsonpickle/unpickler.py
index 283d54b..06864e9 100644
--- a/jsonpickle/unpickler.py
+++ b/jsonpickle/unpickler.py
@@ -606,13 +606,21 @@ def loadclass(module_and_name, classes=None):
except KeyError:
pass
# Otherwise, load classes from globally-accessible imports
- try:
- module, name = module_and_name.rsplit('.', 1)
- module = util.untranslate_module_name(module)
- __import__(module)
- return getattr(sys.modules[module], name)
- except (AttributeError, ImportError, ValueError):
- return None
+ names = module_and_name.split('.')
+ # First assume that everything up to the last dot is the module name,
+ # then try other splits to handle classes that are defined within
+ # classes
+ for up_to in range(len(names)-1, 0, -1):
+ module = util.untranslate_module_name('.'.join(names[:up_to]))
+ try:
+ __import__(module)
+ obj = sys.modules[module]
+ for class_name in names[up_to:]:
+ obj = getattr(obj, class_name)
+ return obj
+ except (AttributeError, ImportError, ValueError) as ex:
+ continue
+ return NoneWith these changes, my example provided above and the examples provided by @ldesegur and @edjubuh work fine. For the second issue, the problem is relatively obvious, the output of this code: import jsonpickle
print(jsonpickle.encode(dir))is Here are the changes that seem to fix this issue: diff --git a/jsonpickle/util.py b/jsonpickle/util.py
index dff7dac..ed6bd8d 100644
--- a/jsonpickle/util.py
+++ b/jsonpickle/util.py
@@ -110,7 +110,8 @@ def is_object(obj):
False
"""
return (isinstance(obj, object) and
- not isinstance(obj, (type, types.FunctionType)))
+ not isinstance(obj, (type, types.FunctionType,
+ types.BuiltinFunctionType)))
def is_primitive(obj):
@@ -277,7 +278,7 @@ def is_module_function(obj):
"""
return (hasattr(obj, '__class__') and
- isinstance(obj, types.FunctionType) and
+ isinstance(obj, (types.FunctionType, types.BuiltinFunctionType)) and
hasattr(obj, '__module__') and
hasattr(obj, '__name__') and
obj.__name__ != '<lambda>')The above example now prints I'd be happy to provide these changes as a pull request (adding/adapting the tests), but I'm unsure what to do about the fact that |
|
This sounds great, yeah I'd totally merge these changes. Regarding the |
I upgraded to version 0.96 after following this issue #147 and KNeibhoursClassifier model still doesn't decode. Here's what I tried-
and the error-
TypeError: 'dict' object is not callableWhat should be done ?
The text was updated successfully, but these errors were encountered: