clean nan/inf in json_clean #2194

merged 2 commits into from Jul 27, 2012


None yet

4 participants

IPython member

these values are not json-compliant

closes #2187

@minrk minrk clean nan/inf in json_clean
these values are not json-compliant
@bfroehle bfroehle commented on an outdated diff Jul 23, 2012
@@ -30,6 +30,12 @@
+# float constants
+NAN = float('nan')
+INF = float('inf')
+NINF = float('-inf')
bfroehle Jul 23, 2012

Is it worth defining all of these if we'll only use INFS in the end?

@bfroehle bfroehle and 1 other commented on an outdated diff Jul 23, 2012
# containers that we need to convert into lists
container_to_list = (tuple, set, types.GeneratorType)
+ if isinstance(obj, float):
+ # cast out-of-range floats to their reprs
+ if obj != obj or obj in INFS:
+ return repr(obj)
bfroehle Jul 23, 2012

You and I both know obj != obj is the way that you test for a a float being NaN, but I wonder if it wouldn't be a lot clearer to do:

if math.isnan(obj) or math.isinf(obj):
    return repr(obj)
minrk Jul 23, 2012 IPython member

I used the exact same tests in stlib json, but those seem clearer, so I will do that.

minrk Jul 23, 2012 IPython member


IPython member

Can we add datetime objects to the list of things to clean? I believe json not handling python datetime objects is the sole reason for these lines that cut out some datetime objects from messages:

@ellisonbg: am I correct in assuming that the reason for popping those dates is because of JSONing limitations?

Update: never mind; I guess I didn't see the rest of the file. There already are functions there to clean dates.

I'm curious why we don't just embed these sorts of things in our own json dumps/loads functions, so you can just call the custom dumps function and it will automatically take care of things like encoding for inf, encoding dates, etc. Python lets you define custom encoders and decoders:

IPython member
IPython member
IPython member

So can we just make a custom JSON encoding class that does both the effects of json_clean and default=date_default automatically? Then it's as easy as calling our JSON encoding function with no arguments.

IPython member

(see for some more ideas how to do datetimes. Especially the suggestion there to use .isoformat() instead of doing your own iso formatting.)

IPython member

Hm, I wonder how I missed .isoformat(). Other than that, I don't see anything preferable in the SO question.

One advantage of decoupling cleaning / datetime-formatting: If a message gets to the Session object that is not JSON-serializable, that is a bug and should raise. Most datetime objects are actually created inside the Session object as the timestamp in every header.

@bfroehle bfroehle merged commit 157d99a into ipython:master Jul 27, 2012

I just committed this NaN/Inf JSON fix. If we want to pursue a custom JSON encoding class that can be done in another issue. (I have NOT opened another issue to track that idea. Perhaps @jasongrout can do it?)

@minrk minrk added a commit that referenced this pull request Jul 28, 2012
@minrk minrk Backport PR #2194: clean nan/inf in json_clean
these values are not json-compliant

closes #2187
@minrk minrk deleted the minrk:clean_nan branch Mar 31, 2014
@yarikoptic yarikoptic added a commit to yarikoptic/ipython that referenced this pull request May 2, 2014
@yarikoptic yarikoptic Merge commit 'rel-0.13-33-gcfc5692' into debian-01X
* commit 'rel-0.13-33-gcfc5692': (33 commits)
  Backport PR #2347: adjust division error message checking to account for Python 3
  Backport PR #2305: RemoteError._render_traceback_ calls self.render_traceback
  Backport PR #2280: fix SSH passwordless check for OpenSSH
  Backport PR #2270: SSHLauncher tweaks
  Backport PR #2261: Fix: longest_substr([]) -> ''
  Backport PR #2250: fix html in notebook example
  Backport PR #2235: remove spurious print statement from
  Backport PR #2223: Custom tracebacks
  Backport PR #2214: use KernelApp.exec_lines/files in IPEngineApp
  Backport PR #2212: catch errors in markdown javascript
  Backport PR #2194: clean nan/inf in json_clean
  Backport PR #2177: remove numpy install from travis/tox scripts
  Backport PR #2169: ipdb: pdef, pdoc, pinfo magics all broken
  Backport PR #2186: removed references to h5py dependence in octave magic documentation
  Backport PR #2185: added test for %store, fixed storemagic
  Backport PR #2170: Fix tab completion with IPython.embed_kernel().
  Backport PR #2163: fix 'remote_profie_dir' typo in SSH launchers
  Backport PR #2117: use explicit url in notebook example
  Backport PR #2126: ipcluster broken with any batch (PBS/LSF/SGE)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment