Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Initial import. Basically everything except the CLI is implemented now.

  • Loading branch information...
commit 2cb15bcbd43c71143c540e1ba21380e480991239 0 parents
@zacharyvoase zacharyvoase authored
11 .gitignore
@@ -0,0 +1,11 @@
+*.egg-info
+*.pyc
+*.pyo
+.DS_Store
+build
+dist
+MANIFEST
+test/example/*.sqlite3
+doc/.build
+distribute-*.egg
+distribute-*.tar.gz
2  MANIFEST.in
@@ -0,0 +1,2 @@
+include distribute_setup.py
+include VERSION
127 README.rst
@@ -0,0 +1,127 @@
+========
+jsonpipe
+========
+
+Everyone I know prefers to work with JSON over XML, but sadly there is a sore
+lack of utilities of the quality or depth of `html-xml-utils`_ and
+`XMLStarlet`_ for actually processing JSON data in an automated fashion, short
+of writing an *ad hoc* processor in your favourite programming language.
+
+.. _html-xml-utils: http://www.w3.org/Tools/HTML-XML-utils/README
+.. _XMLStarlet: http://xmlstar.sourceforge.net/
+
+**jsonpipe** is a step towards a solution: it traverses a JSON object and
+produces a simple, line-based textual format which can be processed by all your
+UNIX favourites like grep, sed, awk, cut and diff. It may also be valuable
+within programming languages---in fact, it was originally conceived as a way of
+writing simple test assertions against JSON output without coupling the tests
+too closely to the specific structure used.
+
+This implementation (which should be considered the reference) is written in
+Python.
+
+
+Example
+=======
+
+A ``<pre>`` is worth a thousand words. For simple JSON values:
+
+ $ echo '"Hello, World!"' | jsonpipe
+ / "Hello, World!"
+ $ echo 123 | jsonpipe
+ / 123
+ $ echo 0.25 | jsonpipe
+ / 0.25
+ $ echo null | jsonpipe
+ / null
+ $ echo true | jsonpipe
+ / true
+ $ echo false | jsonpipe
+ / false
+
+The 'root' of the object tree is represented by a single ``/`` character, and
+for simple values it doesn't get any more complex than the above. Note that a
+single tab character separates the path on the left from the literal value on
+the right.
+
+Composite data structures use a hierarchical syntax, where individual
+keys/indices are children of the path to the containing object:
+
+ $ echo '{"a": 1, "b": 2}' | jsonpipe
+ / {}
+ /a 1
+ /b 2
+ $ echo '["foo", "bar", "baz"]' | jsonpipe
+ / []
+ /0 "foo"
+ /1 "bar"
+ /2 "baz"
+
+For an object or array, the right-hand column indicates the datatype, and will
+be either ``{}`` (object) or ``[]`` (array). For objects, the order of the keys
+is preserved in the output.
+
+The path syntax allows arbitrarily complex data structures:
+
+ $ echo '[{"a": [{"b": {"c": ["foo"]}}]}]' | jsonpipe
+ / []
+ /0 {}
+ /0/a []
+ /0/a/0 {}
+ /0/a/0/b {}
+ /0/a/0/b/c []
+ /0/a/0/b/c/0 "foo"
+
+
+Caveat: Path Separators
+=======================
+
+Because the path components are separated by ``/`` characters, an object key
+like ``"abc/def"`` would result in ambiguous output. jsonpipe will throw
+an error if this occurs in your input, so that you can recognize and handle the
+issue. To mitigate the problem, you can choose a different path separator:
+
+ $ echo '{"abc/def": 123}' | jsonpipe -s '☃'
+ ☃ {}
+ ☃abc/def 123
+
+The Unicode snowman is chosen here because it's unlikely to occur as part of
+the key in most JSON objects, but any character or string (e.g. ``:``, ``::``,
+``~``) will do.
+
+
+Installation
+============
+
+**jsonpipe** is written in Python, so is best installed using ``pip``:
+
+ pip install jsonpipe
+
+Note that it probably requires Python v2.5+ for now, although work on
+compatibility with previous versions of Python is in progress.
+
+
+(Un)license
+===========
+
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or distribute this
+software, either in source code form or as a compiled binary, for any purpose,
+commercial or non-commercial, and by any means.
+
+In jurisdictions that recognize copyright laws, the author or authors of this
+software dedicate any and all copyright interest in the software to the public
+domain. We make this dedication for the benefit of the public at large and to
+the detriment of our heirs and successors. We intend this dedication to be an
+overt act of relinquishment in perpetuity of all present and future rights to
+this software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <http://unlicense.org/>
24 UNLICENSE
@@ -0,0 +1,24 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <http://unlicense.org/>
1  VERSION
@@ -0,0 +1 @@
+0.0.0
485 distribute_setup.py
@@ -0,0 +1,485 @@
+#!python
+"""Bootstrap distribute installation
+
+If you want to use setuptools in your package's setup.py, just include this
+file in the same directory with it, and add this to the top of your setup.py::
+
+ from distribute_setup import use_setuptools
+ use_setuptools()
+
+If you want to require a specific version of setuptools, set a download
+mirror, or use an alternate download directory, you can do so by supplying
+the appropriate options to ``use_setuptools()``.
+
+This file can also be run as a script to install or upgrade setuptools.
+"""
+import os
+import sys
+import time
+import fnmatch
+import tempfile
+import tarfile
+from distutils import log
+
+try:
+ from site import USER_SITE
+except ImportError:
+ USER_SITE = None
+
+try:
+ import subprocess
+
+ def _python_cmd(*args):
+ args = (sys.executable,) + args
+ return subprocess.call(args) == 0
+
+except ImportError:
+ # will be used for python 2.3
+ def _python_cmd(*args):
+ args = (sys.executable,) + args
+ # quoting arguments if windows
+ if sys.platform == 'win32':
+ def quote(arg):
+ if ' ' in arg:
+ return '"%s"' % arg
+ return arg
+ args = [quote(arg) for arg in args]
+ return os.spawnl(os.P_WAIT, sys.executable, *args) == 0
+
+DEFAULT_VERSION = "0.6.14"
+DEFAULT_URL = "http://pypi.python.org/packages/source/d/distribute/"
+SETUPTOOLS_FAKED_VERSION = "0.6c11"
+
+SETUPTOOLS_PKG_INFO = """\
+Metadata-Version: 1.0
+Name: setuptools
+Version: %s
+Summary: xxxx
+Home-page: xxx
+Author: xxx
+Author-email: xxx
+License: xxx
+Description: xxx
+""" % SETUPTOOLS_FAKED_VERSION
+
+
+def _install(tarball):
+ # extracting the tarball
+ tmpdir = tempfile.mkdtemp()
+ log.warn('Extracting in %s', tmpdir)
+ old_wd = os.getcwd()
+ try:
+ os.chdir(tmpdir)
+ tar = tarfile.open(tarball)
+ _extractall(tar)
+ tar.close()
+
+ # going in the directory
+ subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
+ os.chdir(subdir)
+ log.warn('Now working in %s', subdir)
+
+ # installing
+ log.warn('Installing Distribute')
+ if not _python_cmd('setup.py', 'install'):
+ log.warn('Something went wrong during the installation.')
+ log.warn('See the error message above.')
+ finally:
+ os.chdir(old_wd)
+
+
+def _build_egg(egg, tarball, to_dir):
+ # extracting the tarball
+ tmpdir = tempfile.mkdtemp()
+ log.warn('Extracting in %s', tmpdir)
+ old_wd = os.getcwd()
+ try:
+ os.chdir(tmpdir)
+ tar = tarfile.open(tarball)
+ _extractall(tar)
+ tar.close()
+
+ # going in the directory
+ subdir = os.path.join(tmpdir, os.listdir(tmpdir)[0])
+ os.chdir(subdir)
+ log.warn('Now working in %s', subdir)
+
+ # building an egg
+ log.warn('Building a Distribute egg in %s', to_dir)
+ _python_cmd('setup.py', '-q', 'bdist_egg', '--dist-dir', to_dir)
+
+ finally:
+ os.chdir(old_wd)
+ # returning the result
+ log.warn(egg)
+ if not os.path.exists(egg):
+ raise IOError('Could not build the egg.')
+
+
+def _do_download(version, download_base, to_dir, download_delay):
+ egg = os.path.join(to_dir, 'distribute-%s-py%d.%d.egg'
+ % (version, sys.version_info[0], sys.version_info[1]))
+ if not os.path.exists(egg):
+ tarball = download_setuptools(version, download_base,
+ to_dir, download_delay)
+ _build_egg(egg, tarball, to_dir)
+ sys.path.insert(0, egg)
+ import setuptools
+ setuptools.bootstrap_install_from = egg
+
+
+def use_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
+ to_dir=os.curdir, download_delay=15, no_fake=True):
+ # making sure we use the absolute path
+ to_dir = os.path.abspath(to_dir)
+ was_imported = 'pkg_resources' in sys.modules or \
+ 'setuptools' in sys.modules
+ try:
+ try:
+ import pkg_resources
+ if not hasattr(pkg_resources, '_distribute'):
+ if not no_fake:
+ _fake_setuptools()
+ raise ImportError
+ except ImportError:
+ return _do_download(version, download_base, to_dir, download_delay)
+ try:
+ pkg_resources.require("distribute>="+version)
+ return
+ except pkg_resources.VersionConflict:
+ e = sys.exc_info()[1]
+ if was_imported:
+ sys.stderr.write(
+ "The required version of distribute (>=%s) is not available,\n"
+ "and can't be installed while this script is running. Please\n"
+ "install a more recent version first, using\n"
+ "'easy_install -U distribute'."
+ "\n\n(Currently using %r)\n" % (version, e.args[0]))
+ sys.exit(2)
+ else:
+ del pkg_resources, sys.modules['pkg_resources'] # reload ok
+ return _do_download(version, download_base, to_dir,
+ download_delay)
+ except pkg_resources.DistributionNotFound:
+ return _do_download(version, download_base, to_dir,
+ download_delay)
+ finally:
+ if not no_fake:
+ _create_fake_setuptools_pkg_info(to_dir)
+
+def download_setuptools(version=DEFAULT_VERSION, download_base=DEFAULT_URL,
+ to_dir=os.curdir, delay=15):
+ """Download distribute from a specified location and return its filename
+
+ `version` should be a valid distribute version number that is available
+ as an egg for download under the `download_base` URL (which should end
+ with a '/'). `to_dir` is the directory where the egg will be downloaded.
+ `delay` is the number of seconds to pause before an actual download
+ attempt.
+ """
+ # making sure we use the absolute path
+ to_dir = os.path.abspath(to_dir)
+ try:
+ from urllib.request import urlopen
+ except ImportError:
+ from urllib2 import urlopen
+ tgz_name = "distribute-%s.tar.gz" % version
+ url = download_base + tgz_name
+ saveto = os.path.join(to_dir, tgz_name)
+ src = dst = None
+ if not os.path.exists(saveto): # Avoid repeated downloads
+ try:
+ log.warn("Downloading %s", url)
+ src = urlopen(url)
+ # Read/write all in one block, so we don't create a corrupt file
+ # if the download is interrupted.
+ data = src.read()
+ dst = open(saveto, "wb")
+ dst.write(data)
+ finally:
+ if src:
+ src.close()
+ if dst:
+ dst.close()
+ return os.path.realpath(saveto)
+
+def _no_sandbox(function):
+ def __no_sandbox(*args, **kw):
+ try:
+ from setuptools.sandbox import DirectorySandbox
+ if not hasattr(DirectorySandbox, '_old'):
+ def violation(*args):
+ pass
+ DirectorySandbox._old = DirectorySandbox._violation
+ DirectorySandbox._violation = violation
+ patched = True
+ else:
+ patched = False
+ except ImportError:
+ patched = False
+
+ try:
+ return function(*args, **kw)
+ finally:
+ if patched:
+ DirectorySandbox._violation = DirectorySandbox._old
+ del DirectorySandbox._old
+
+ return __no_sandbox
+
+def _patch_file(path, content):
+ """Will backup the file then patch it"""
+ existing_content = open(path).read()
+ if existing_content == content:
+ # already patched
+ log.warn('Already patched.')
+ return False
+ log.warn('Patching...')
+ _rename_path(path)
+ f = open(path, 'w')
+ try:
+ f.write(content)
+ finally:
+ f.close()
+ return True
+
+_patch_file = _no_sandbox(_patch_file)
+
+def _same_content(path, content):
+ return open(path).read() == content
+
+def _rename_path(path):
+ new_name = path + '.OLD.%s' % time.time()
+ log.warn('Renaming %s into %s', path, new_name)
+ os.rename(path, new_name)
+ return new_name
+
+def _remove_flat_installation(placeholder):
+ if not os.path.isdir(placeholder):
+ log.warn('Unkown installation at %s', placeholder)
+ return False
+ found = False
+ for file in os.listdir(placeholder):
+ if fnmatch.fnmatch(file, 'setuptools*.egg-info'):
+ found = True
+ break
+ if not found:
+ log.warn('Could not locate setuptools*.egg-info')
+ return
+
+ log.warn('Removing elements out of the way...')
+ pkg_info = os.path.join(placeholder, file)
+ if os.path.isdir(pkg_info):
+ patched = _patch_egg_dir(pkg_info)
+ else:
+ patched = _patch_file(pkg_info, SETUPTOOLS_PKG_INFO)
+
+ if not patched:
+ log.warn('%s already patched.', pkg_info)
+ return False
+ # now let's move the files out of the way
+ for element in ('setuptools', 'pkg_resources.py', 'site.py'):
+ element = os.path.join(placeholder, element)
+ if os.path.exists(element):
+ _rename_path(element)
+ else:
+ log.warn('Could not find the %s element of the '
+ 'Setuptools distribution', element)
+ return True
+
+_remove_flat_installation = _no_sandbox(_remove_flat_installation)
+
+def _after_install(dist):
+ log.warn('After install bootstrap.')
+ placeholder = dist.get_command_obj('install').install_purelib
+ _create_fake_setuptools_pkg_info(placeholder)
+
+def _create_fake_setuptools_pkg_info(placeholder):
+ if not placeholder or not os.path.exists(placeholder):
+ log.warn('Could not find the install location')
+ return
+ pyver = '%s.%s' % (sys.version_info[0], sys.version_info[1])
+ setuptools_file = 'setuptools-%s-py%s.egg-info' % \
+ (SETUPTOOLS_FAKED_VERSION, pyver)
+ pkg_info = os.path.join(placeholder, setuptools_file)
+ if os.path.exists(pkg_info):
+ log.warn('%s already exists', pkg_info)
+ return
+
+ log.warn('Creating %s', pkg_info)
+ f = open(pkg_info, 'w')
+ try:
+ f.write(SETUPTOOLS_PKG_INFO)
+ finally:
+ f.close()
+
+ pth_file = os.path.join(placeholder, 'setuptools.pth')
+ log.warn('Creating %s', pth_file)
+ f = open(pth_file, 'w')
+ try:
+ f.write(os.path.join(os.curdir, setuptools_file))
+ finally:
+ f.close()
+
+_create_fake_setuptools_pkg_info = _no_sandbox(_create_fake_setuptools_pkg_info)
+
+def _patch_egg_dir(path):
+ # let's check if it's already patched
+ pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
+ if os.path.exists(pkg_info):
+ if _same_content(pkg_info, SETUPTOOLS_PKG_INFO):
+ log.warn('%s already patched.', pkg_info)
+ return False
+ _rename_path(path)
+ os.mkdir(path)
+ os.mkdir(os.path.join(path, 'EGG-INFO'))
+ pkg_info = os.path.join(path, 'EGG-INFO', 'PKG-INFO')
+ f = open(pkg_info, 'w')
+ try:
+ f.write(SETUPTOOLS_PKG_INFO)
+ finally:
+ f.close()
+ return True
+
+_patch_egg_dir = _no_sandbox(_patch_egg_dir)
+
+def _before_install():
+ log.warn('Before install bootstrap.')
+ _fake_setuptools()
+
+
+def _under_prefix(location):
+ if 'install' not in sys.argv:
+ return True
+ args = sys.argv[sys.argv.index('install')+1:]
+ for index, arg in enumerate(args):
+ for option in ('--root', '--prefix'):
+ if arg.startswith('%s=' % option):
+ top_dir = arg.split('root=')[-1]
+ return location.startswith(top_dir)
+ elif arg == option:
+ if len(args) > index:
+ top_dir = args[index+1]
+ return location.startswith(top_dir)
+ if arg == '--user' and USER_SITE is not None:
+ return location.startswith(USER_SITE)
+ return True
+
+
+def _fake_setuptools():
+ log.warn('Scanning installed packages')
+ try:
+ import pkg_resources
+ except ImportError:
+ # we're cool
+ log.warn('Setuptools or Distribute does not seem to be installed.')
+ return
+ ws = pkg_resources.working_set
+ try:
+ setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools',
+ replacement=False))
+ except TypeError:
+ # old distribute API
+ setuptools_dist = ws.find(pkg_resources.Requirement.parse('setuptools'))
+
+ if setuptools_dist is None:
+ log.warn('No setuptools distribution found')
+ return
+ # detecting if it was already faked
+ setuptools_location = setuptools_dist.location
+ log.warn('Setuptools installation detected at %s', setuptools_location)
+
+ # if --root or --preix was provided, and if
+ # setuptools is not located in them, we don't patch it
+ if not _under_prefix(setuptools_location):
+ log.warn('Not patching, --root or --prefix is installing Distribute'
+ ' in another location')
+ return
+
+ # let's see if its an egg
+ if not setuptools_location.endswith('.egg'):
+ log.warn('Non-egg installation')
+ res = _remove_flat_installation(setuptools_location)
+ if not res:
+ return
+ else:
+ log.warn('Egg installation')
+ pkg_info = os.path.join(setuptools_location, 'EGG-INFO', 'PKG-INFO')
+ if (os.path.exists(pkg_info) and
+ _same_content(pkg_info, SETUPTOOLS_PKG_INFO)):
+ log.warn('Already patched.')
+ return
+ log.warn('Patching...')
+ # let's create a fake egg replacing setuptools one
+ res = _patch_egg_dir(setuptools_location)
+ if not res:
+ return
+ log.warn('Patched done.')
+ _relaunch()
+
+
+def _relaunch():
+ log.warn('Relaunching...')
+ # we have to relaunch the process
+ # pip marker to avoid a relaunch bug
+ if sys.argv[:3] == ['-c', 'install', '--single-version-externally-managed']:
+ sys.argv[0] = 'setup.py'
+ args = [sys.executable] + sys.argv
+ sys.exit(subprocess.call(args))
+
+
+def _extractall(self, path=".", members=None):
+ """Extract all members from the archive to the current working
+ directory and set owner, modification time and permissions on
+ directories afterwards. `path' specifies a different directory
+ to extract to. `members' is optional and must be a subset of the
+ list returned by getmembers().
+ """
+ import copy
+ import operator
+ from tarfile import ExtractError
+ directories = []
+
+ if members is None:
+ members = self
+
+ for tarinfo in members:
+ if tarinfo.isdir():
+ # Extract directories with a safe mode.
+ directories.append(tarinfo)
+ tarinfo = copy.copy(tarinfo)
+ tarinfo.mode = 448 # decimal for oct 0700
+ self.extract(tarinfo, path)
+
+ # Reverse sort directories.
+ if sys.version_info < (2, 4):
+ def sorter(dir1, dir2):
+ return cmp(dir1.name, dir2.name)
+ directories.sort(sorter)
+ directories.reverse()
+ else:
+ directories.sort(key=operator.attrgetter('name'), reverse=True)
+
+ # Set correct owner, mtime and filemode on directories.
+ for tarinfo in directories:
+ dirpath = os.path.join(path, tarinfo.name)
+ try:
+ self.chown(tarinfo, dirpath)
+ self.utime(tarinfo, dirpath)
+ self.chmod(tarinfo, dirpath)
+ except ExtractError:
+ e = sys.exc_info()[1]
+ if self.errorlevel > 1:
+ raise
+ else:
+ self._dbg(1, "tarfile: %s" % e)
+
+
+def main(argv, version=DEFAULT_VERSION):
+ """Install or upgrade setuptools and EasyInstall"""
+ tarball = download_setuptools()
+ _install(tarball)
+
+
+if __name__ == '__main__':
+ main(sys.argv[1:])
24 setup.py
@@ -0,0 +1,24 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+from distribute_setup import use_setuptools
+use_setuptools()
+
+from setuptools import setup
+import os.path as p
+
+VERSION = open(p.join(p.dirname(p.abspath(__file__)), 'VERSION')).read().strip()
+
+setup(
+ name='jsonpipe',
+ version=VERSION,
+ description="Convert JSON to a UNIX-friendly line-based format.",
+ author='Zachary Voase',
+ author_email='z@dvxhouse.com',
+ url='http://github.com/dvxhouse/jsonpipe',
+ package_dir={'': 'src'},
+ py_modules=['jsonpipe'],
+ entry_points={'console_scripts': ['jsonpipe = jsonpipe:main']},
+ install_requires=['simplejson>=2.1.3'],
+ test_suite='jsonpipe._get_tests',
+)
246 src/jsonpipe.py
@@ -0,0 +1,246 @@
+# -*- coding: utf-8 -*-
+
+from __future__ import with_statement
+import contextlib
+import sys
+
+import simplejson
+
+
+class JSONPiper(object):
+
+ u"""
+ Class to convert a (parsed) JSON object into a UNIX-friendly text stream.
+
+ You need to initialize your piper with a stream; for practical purposes
+ I'll be using `sys.stdout`:
+
+ >>> import sys
+ >>> p = JSONPiper(sys.stdout)
+
+ Please note that, where applicable, *all* input must use either native
+ Unicode strings or UTF-8-encoded bytestrings, and all output will be UTF-8
+ encoded.
+
+ The simplest case is outputting JSON values (strings, numbers, booleans and
+ nulls):
+
+ >>> p.write(u"Hello, World!")
+ / "Hello, World!"
+ >>> p.write(123)
+ / 123
+ >>> p.write(0.25)
+ / 0.25
+ >>> p.write(None)
+ / null
+ >>> p.write(True)
+ / true
+ >>> p.write(False)
+ / false
+
+ jsonpipe always uses '/' to represent the top-level object. Dictionaries
+ are displayed as ``{}``, with each key shown as a sub-path:
+
+ >>> p.write({"a": 1, "b": 2})
+ / {}
+ /a 1
+ /b 2
+
+ Lists are treated in much the same way, only the integer indices are used
+ as the keys, and the top-level list object is shown as ``[]``:
+
+ >>> p.write([1, "foo", 2, "bar"])
+ / []
+ /0 1
+ /1 "foo"
+ /2 2
+ /3 "bar"
+
+ Finally, the practical benefit of using hierarchical paths is that the
+ syntax supports nesting of arbitrarily complex constructs:
+
+ >>> p.write([{"a": [{"b": {"c": ["foo"]}}]}])
+ / []
+ /0 {}
+ /0/a []
+ /0/a/0 {}
+ /0/a/0/b {}
+ /0/a/0/b/c []
+ /0/a/0/b/c/0 "foo"
+
+ Because the sole separator of path components is a ``/`` character by
+ default, keys containing this character would result in ambiguous output.
+ Therefore, if you try to write a dictionary with a key containing the path
+ separator, `JSONPiper` will raise a :exc:`ValueError`:
+
+ >>> p.write({"a/b": 1})
+ Traceback (most recent call last):
+ ...
+ ValueError: Path separator '/' present in key 'a/b'
+
+ In more complex examples, some output may be written before the exception
+ is raised. To mitigate this problem, you can provide a custom path
+ separator:
+
+ >>> colon_p = JSONPiper(sys.stdout, pathsep=':')
+ >>> colon_p.write({"a/b": 1})
+ : {}
+ :a/b 1
+
+ The path separator should be a bytestring, and it is of course advisable
+ that you use something you are almost certain will not be present in your
+ dictionary keys.
+ """
+
+ def __init__(self, stream, pathsep='/'):
+ self.stream = stream
+ self.stack = []
+ self.pathsep = pathsep
+
+ def write(self, obj):
+
+ """
+ Output the provided (parsed) JSON object to this piper's stream.
+
+ The object can be as complex as you like, but it must consist only of:
+
+ * Dictionaries (or subclasses of `dict`)
+ * Lists or tuples (or subclasses of the built-in types)
+ * Unicode Strings (`unicode`, utf-8 encoded `str`)
+ * Numbers (`int`, `long`, `float`)
+ * Booleans (`True`, `False`)
+ * `None`
+
+ Consult the :class:`JSONPiper` documentation for a full run-down of the
+ output format.
+ """
+
+ if is_value(obj):
+ self._write(simplejson.dumps(obj))
+ return
+
+ if isinstance(obj, dict):
+ self._write('{}')
+ iterator = obj.iteritems()
+ elif isinstance(obj, (list, tuple)):
+ self._write('[]')
+ iterator = enumerate(obj)
+ else:
+ raise TypeError("Unsupported type for jsonpipe output: %r" % type(obj))
+
+ for key, value in iterator:
+ with self.push(key) as sub_writer:
+ sub_writer.write(value)
+
+ def _write(self, string):
+ """Write a simple string to the output stream at the current path."""
+
+ self.stream.write("%s\t%s\n" % (self.path, string))
+
+ @contextlib.contextmanager
+ def push(self, key):
+
+ """
+ Context manager to push a key onto the stack and pop it afterwards.
+
+ For now, this will modify the piper inside the `with` block and reverse
+ the modification afterwards, but in future it may return a new piper
+ object, so you should use the bound variable in the body of your `with`
+ block for forwards-compatibility.
+
+ Example:
+
+ >>> import sys
+ >>> p = JSONPiper(sys.stdout)
+ >>> print p.path
+ /
+ >>> with p.push('example') as p2:
+ ... print p2.path
+ /example
+ >>> print p.path
+ /
+ """
+
+ key = to_str(key)
+ if self.pathsep in key:
+ # In almost any case this is not what the user wants; because
+ # joining the strings with a human-readable character is
+ # potentially a lossy process we should fail if
+ raise ValueError("Path separator %r present in key %r" %
+ (self.pathsep, key))
+
+ self.stack.append(key)
+ try:
+ yield self
+ finally:
+ self.stack.pop()
+
+ @property
+ def path(self):
+ """The current path in the object tree, as a bytestring."""
+
+ return self.pathsep + self.pathsep.join(self.stack)
+
+
+def to_str(obj):
+
+ ur"""
+ Coerce an object to a bytestring, utf-8-encoding if necessary.
+
+ >>> to_str("Hello World")
+ 'Hello World'
+ >>> to_str(u"H\xe9llo")
+ 'H\xc3\xa9llo'
+ """
+
+ if isinstance(obj, unicode):
+ return obj.encode('utf-8')
+ elif hasattr(obj, '__unicode__'):
+ return unicode(obj).encode('utf-8')
+ return str(obj)
+
+
+def is_value(obj):
+
+ """
+ Determine whether an object is a simple JSON value.
+
+ The phrase 'simple JSON value' here means one of:
+
+ * String (Unicode or UTF-8-encoded bytestring)
+ * Number (integer or floating-point)
+ * Boolean
+ * `None`
+ """
+
+ return isinstance(obj, (str, unicode, int, long, float, bool, type(None)))
+
+
+def jsonpipe(json_obj, stream=None, writer=None):
+
+ """
+ Pipe the provided JSON object to the output stream (defaulting to stdout).
+
+ This is just a shortcut for constructing a :class:`JSONPiper`:
+
+ >>> jsonpipe({"a": 1, "b": 2})
+ / {}
+ /a 1
+ /b 2
+ """
+
+ if stream is None:
+ stream = sys.stdout
+ if writer is None:
+ writer = JSONPiper(stream)
+ writer.write(json_obj)
+
+
+def _get_tests():
+ import doctest
+ return doctest.DocTestSuite(
+ optionflags=(doctest.ELLIPSIS | doctest.NORMALIZE_WHITESPACE))
+
+
+def main():
+ raise NotImplementedError
Please sign in to comment.
Something went wrong with that request. Please try again.