JSON decoder timestamp error #170

durack1 · 2017-04-06T19:59:50Z

It seems that a json package update has triggered the following warning, when vcs imports this package it seems to require a global attribute or environment variable set that isn't currently set

In [1]: import vcs
/export/durack1/anaconda2/envs/uvcdat280/lib/python2.7/json/encoder.py:207: DeprecationWarning: Interpreting naive datetime as local 2017-04-06 12:57:51.759099. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)
/export/durack1/anaconda2/envs/uvcdat280/lib/python2.7/json/encoder.py:207: DeprecationWarning: Interpreting naive datetime as local 2017-04-06 12:57:53.901765. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)
/export/durack1/anaconda2/envs/uvcdat280/lib/python2.7/json/encoder.py:207: DeprecationWarning: Interpreting naive datetime as local 2017-04-06 12:57:53.912374. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)

The text was updated successfully, but these errors were encountered:

durack1 · 2017-04-11T21:20:30Z

@doutriaux1 this was the issue that I pointed out yesterday was making it very difficult to work interactively.. It seems the import statement for the json library needs an additional timezone (or similar) argument to solve this

durack1 · 2017-04-11T21:25:09Z

@doutriaux1 this might work as implemented for jupyter:

def _ensure_tzinfo(dt):
    """Ensure a datetime object has tzinfo
    
    If no tzinfo is present, add tzlocal
    """
    if not dt.tzinfo:
        # No more naïve datetime objects!
        warnings.warn(u"Interpreting naive datetime as local %s. Please add timezone info to timestamps." % dt,
            DeprecationWarning,
            stacklevel=4)
        dt = dt.replace(tzinfo=tzlocal())
return dt

durack1 · 2017-04-13T00:12:48Z

@doutriaux1 this is now rendering vcs unusable in a console, it just locks the console on some infinite loop or similar:

In [10]: import vcs
/export/durack1/anaconda2/envs/cdatcmornclnco/lib/python2.7/json/encoder.py:207: DeprecationWarning: Interpreting naive datetime as local 2017-04-12 17:11:19.496976. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)
In [11]: /export/durack1/anaconda2/envs/cdatcmornclnco/lib/python2.7/json/encoder.py:207: DeprecationWarning: Interpreting naive datetime as local 2017-04-12 17:11:21.349779. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)
/export/durack1/anaconda2/envs/cdatcmornclnco/lib/python2.7/json/encoder.py:207: DeprecationWarning: Interpreting naive datetime as local 2017-04-12 17:11:21.370540. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)




no more active python session

doutriaux1 · 2017-04-13T14:09:36Z

@durack1 on oceanonly? Can you please provide

json version you're using
sample script

thx!

durack1 · 2017-04-13T15:42:27Z

@doutriaux1 it's wrapped up in an import statement somewhere, and is occurring when using the spyder IDE - I wasn't able to reproduce this with just ipython.. The json library is a stdlib, so ships with python 2.7.13
So the steps are:

Create a conda environment with cdat
Install spyder into that environment
Open spyder
Import vcs and the DeprecationWarning should appear

doutriaux1 · 2017-04-13T15:57:44Z

ok so it does sound like a spyder issue no?

doutriaux1 · 2017-04-13T15:57:55Z

i'll take a look

durack1 · 2017-04-13T16:00:42Z

@doutriaux1 nope, I tried testing a vanilla spyder-only install and it doesn't seem to trigger anything, it's something on our side I think.. Having said that I commented on a similar issue at spyder-ide/spyder#4196

doutriaux1 · 2017-04-13T16:16:48Z

i don't see what triggers this though. There's no link to our code anywhere, I can try to pdb it....

doutriaux1 · 2017-04-13T16:19:06Z

I really think it's a spyder issue, json is part of python

(nightly) doutriaux1@loki:[pcmdi_metrics]:[334_monsoon_wang_charles_review+3]:[2772]> python -c "import json ; print reload(json)"
<module 'json' from '/Users/doutriaux1/anaconda2/envs/nightly/lib/python2.7/json/__init__.pyc'>

it's their parsing thing that's broken

doutriaux1 · 2017-04-13T16:19:15Z

when did it break?

durack1 · 2017-04-13T16:19:19Z

@doutriaux1 it triggers when vcs is imported, but when I tried to step through each of the init.py entries I couldn't reproduce the error.. It's confusing

durack1 · 2017-04-13T16:19:34Z

@doutriaux1 it broke with the recent builds.. So one of the dependencies changed, and it started to occur

doutriaux1 · 2017-04-13T16:20:49Z

trying with 2.8, the main diff is probably that we refactored all the docstrings

doutriaux1 · 2017-04-13T17:49:48Z

@durack1 please try nightlies I just uploaded on conda and let me know. Thx.

durack1 · 2017-04-13T20:12:04Z

@doutriaux1 did you chase down the issue? Is there a PR somewhere? And which package needs updating from conda?

doutriaux1 · 2017-04-13T21:18:19Z

it's in master and in nightlies please try and report. One of the doc string seemed to have a weird type, but I'm a bit afraid I broke some doc string by fixing it, because it was the only change in a commit titled "fix doc string"

durack1 · 2017-04-13T21:30:17Z

@doutriaux1 which package - vcs? I'll overwrite and start testing this now

doutriaux1 · 2017-04-13T21:49:01Z

vcs

durack1 · 2017-04-14T01:17:16Z

@doutriaux1 I don't suppose there is a -nox version?

durack1 · 2017-04-14T01:26:51Z

@doutriaux1 almost...

Python 2.7.13 | packaged by conda-forge | (default, Mar 20 2017, 14:04:17)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
In [1]: import vcs
Traceback (most recent call last):
  File "<ipython-input-1-94654af38d03>", line 1, in <module>
    import vcs
  File "/export/durack1/anaconda2/envs/uvcdatNightly/lib/python2.7/site-packages/vcs/__init__.py", line 56, in <module>
    from utils import *  # noqa
  File "/export/durack1/anaconda2/envs/uvcdatNightly/lib/python2.7/site-packages/vcs/utils.py", line 19, in <module>
    import boxfill
  File "/export/durack1/anaconda2/envs/uvcdatNightly/lib/python2.7/site-packages/vcs/boxfill.py", line 26, in <module>
    import xmldocs
  File "/export/durack1/anaconda2/envs/uvcdatNightly/lib/python2.7/site-packages/vcs/xmldocs.py", line 884, in <module>
    populate_docstrings(obj_details, docstrings[method][0], docstrings[method][1], method)
  File "/export/durack1/anaconda2/envs/uvcdatNightly/lib/python2.7/site-packages/vcs/xmldocs.py", line 543, in populate_docstrings
    target_dict[obj_name] = docstring % d
KeyError: 'method_type'

And the version info:

[bob@taylor ~]$ conda create -n uvcdatNightly -c conda-forge -c uvcdat/label/nightly -c uvcdat uvcdat
cdat_info:                          2.8.2-py27_0                                         conda-forge
cdms2:                              2.8-np111py27_0                                      conda-forge
vcs:                                2017.4.13.45a557da7342271fffe9e78821b1515568c264dd-0 uvcdat/label/nightly
vtk-cdat:                           7.1.0.2.8-py27_2                                     uvcdat
bob@taylor:[~]:[4506]> source activate uvcdatNightly
(uvcdatNightly) bob@taylor:[~]:[4489]> conda install spyder
spyder:            3.1.3-py27_0

durack1 · 2017-04-14T17:51:25Z

@aashish24 @embrown

grep -R DeprecationWarning vcs/*
vcs/Canvas.py:# Python < 3 DeprecationWarning ignored by default
vcs/Canvas.py:        warnings.warn("Export to GhostScript is no longer supported", DeprecationWarning)
Binary file vcs/Canvas.pyc matches
vcs/boxfill.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/boxfill.pyc matches
vcs/colormap.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/colormap.pyc matches
vcs/dv3d.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/dv3d.pyc matches
vcs/fillarea.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/fillarea.pyc matches
vcs/isofill.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/isofill.pyc matches
vcs/isoline.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/isoline.pyc matches
vcs/line.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/line.pyc matches
vcs/marker.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/marker.pyc matches
vcs/meshfill.py:            raise DeprecationWarning("scr script are no longer generated")
Binary file vcs/meshfill.pyc matches
vcs/projection.py:            raise DeprecationWarning("scr script are no longer generated")
…

durack1 · 2017-04-14T19:17:13Z

@doutriaux1 yep, it worked. Commenting out vcs/Canvas.py#L65

63
64 # Python < 3 DeprecationWarning ignored by default
65 #warnings.simplefilter('default')
66

Leads to an iPython/jupyter session now looking like:

Python 2.7.13 | packaged by conda-forge | (default, Mar 20 2017, 14:04:17)
Type "copyright", "credits" or "license" for more information.
IPython 5.3.0 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.
import vcs
In [2]:

So it's just a matter of checking with @aashish24 and @embrown to make sure this tweak doesn't break anything and I am fixed.. And the issue can be closed

durack1 · 2017-04-14T19:29:35Z

@aashish24 it seems like very bad practice to start changing the default python warnings behavior in any package.. I am very curious why you included the warnings code above (or at least updated it, rather than removing it)

doutriaux1 · 2017-04-14T20:33:06Z

#178 fixes this

ghost · 2017-04-15T15:10:19Z

@doutriaux1 @durack1 this shouldn't break the docstrings, if it's just a deprecation warning being commented out. I never put any warnings in, so I'm not sure whether this was there before I touched it or put in after.

doutriaux1 · 2017-04-18T14:04:55Z

@embrown it's vcs/vtk related we have a bug fix (#178) thanks for chiming in though!

durack1 mentioned this issue Apr 14, 2017

DeprecationWarning in json/encoder.py jupyter/jupyter#253

Closed

doutriaux1 modified the milestone: 3.0 May 5, 2017

doutriaux1 modified the milestones: 3.0, post 3.0 Mar 29, 2018

jzang18 mentioned this issue Oct 26, 2018

how to remove the following DeprecationWarning msg in jupyter notebook after using pygam? dswah/pyGAM#220

Closed

doutriaux1 modified the milestones: 8.1, 8.2 Mar 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

JSON decoder timestamp error #170

JSON decoder timestamp error #170

durack1 commented Apr 6, 2017

durack1 commented Apr 11, 2017

durack1 commented Apr 11, 2017

durack1 commented Apr 13, 2017 •

edited

Loading

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 •

edited

Loading

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 •

edited

Loading

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 •

edited

Loading

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 •

edited

Loading

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 14, 2017

durack1 commented Apr 14, 2017

durack1 commented Apr 14, 2017

durack1 commented Apr 14, 2017 •

edited

Loading

durack1 commented Apr 14, 2017 •

edited

Loading

doutriaux1 commented Apr 14, 2017

ghost commented Apr 15, 2017

doutriaux1 commented Apr 18, 2017

JSON decoder timestamp error #170

JSON decoder timestamp error #170

Comments

durack1 commented Apr 6, 2017

durack1 commented Apr 11, 2017

durack1 commented Apr 11, 2017

durack1 commented Apr 13, 2017 • edited Loading

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 • edited Loading

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 • edited Loading

doutriaux1 commented Apr 13, 2017

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 • edited Loading

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 13, 2017 • edited Loading

doutriaux1 commented Apr 13, 2017

durack1 commented Apr 14, 2017

durack1 commented Apr 14, 2017

durack1 commented Apr 14, 2017

durack1 commented Apr 14, 2017 • edited Loading

durack1 commented Apr 14, 2017 • edited Loading

doutriaux1 commented Apr 14, 2017

ghost commented Apr 15, 2017

doutriaux1 commented Apr 18, 2017

durack1 commented Apr 13, 2017 •

edited

Loading

durack1 commented Apr 13, 2017 •

edited

Loading

durack1 commented Apr 13, 2017 •

edited

Loading

durack1 commented Apr 13, 2017 •

edited

Loading

durack1 commented Apr 13, 2017 •

edited

Loading

durack1 commented Apr 14, 2017 •

edited

Loading

durack1 commented Apr 14, 2017 •

edited

Loading