Skip to content

Commit

Permalink
adding to documentation before release
Browse files Browse the repository at this point in the history
  • Loading branch information
Alex Kovner committed Dec 28, 2018
1 parent 7e49f3b commit 5ea8eca
Show file tree
Hide file tree
Showing 9 changed files with 39 additions and 17 deletions.
2 changes: 1 addition & 1 deletion .travis.yml
Expand Up @@ -28,7 +28,7 @@ jobs:
on:
all_branches: true
- stage: deploy-prod
if: (branch = master) && (tag IS present) && (tag =~ /^v(\d+!)?\d+(\.\d+)*((a|b|rc)\d+)?(\.post\d+)?$/)
if: (tag IS present) && (tag =~ /^v(\d+!)?\d+(\.\d+)*((a|b|rc)\d+)?(\.post\d+)?$/)
python: "3.6"
before_deploy:
- build_scripts/add_version
Expand Down
27 changes: 23 additions & 4 deletions README.rst
Expand Up @@ -16,6 +16,11 @@ documentation

See `<https://jsv.readthedocs.io/>`_

installation
------------

``pip install jsv``

motivation
----------

Expand All @@ -39,7 +44,7 @@ Let's start with a simple example. Suppose we have a list of three json objects
We transform this into: ::

#_ "template":{"key_1","key_2","key_3"})
#_ {"key_1","key_2","key_3"}
{1,2,"three"}
{"four",true,6}
{7,"eight",null}
Expand Down Expand Up @@ -94,7 +99,7 @@ Here is the initial json lines file: ::
{"account_number":111111111,"new_address":{"street":"123 main st.","city":"San Francisco","state":"CA","zip":"94103"}
{"account_number":222222222,"transaction_type":"sale","merchant_id":848757678,"amount":5974.29}
Here is the jsv file: ::
Here is the ``.jsv`` file: ::

#trns {"account_number","transaction_type","merchant_id","amount"}
@trns {111111111,"sale",987654321,123.45}
Expand All @@ -110,6 +115,22 @@ The ``@`` followed by the template name indicates a record. New record types can
@addr {111111111,{"123 main st.","San Francisco","CA","94103"}}
{222222222,:"sale",848757678,5974.29}

split template and record files
+++++++++++++++++++++++++++++++

We can also store the templates in a separate file. By convention, we use the ``.jsvt`` extension for the template file, and the ``.jsvr`` extension for the record file. Using the example from the previous section, here is the template file: ::

#trns {"account_number","transaction_type","merchant_id","amount"}
#addr {"id":"A","name":"address change","template":{"account_number","new_address":{"street","city","state","zip"}}}

and here is the record file: ::

@trns {111111111,"sale",987654321,123.45}
@addr {111111111,{"123 main st.","San Francisco","CA","94103"}}
@trns {222222222,:"sale",848757678,5974.29}

This feature is intended to facilitate analysis on a cluster device, where the record file can be split among nodes, and the template file can be put in the global cache.

definitions
-----------

Expand All @@ -123,8 +144,6 @@ record

object
An ordinary json object, or its equivalent representation in a given language.

In effect, we are converting dictionaries to lists in the values object, but we are careful to distinguish between a list that will be converted back to a dictionary. The same goes for the keys object, except that the primitives are all strings. Any library that implements the jsv format must therefore define list-like data structures to handle these cases.

future features
---------------
Expand Down
3 changes: 1 addition & 2 deletions docs/source/api.rst
Expand Up @@ -18,5 +18,4 @@ Exceptions
----------

.. autoclass:: jsv.JSVTemplateDecodeError
.. autoclass:: jsv.JSVRecordDecodeError
.. autoclass:: jsv.JSVRecordEncodeError
.. autoclass:: jsv.JSVRecordDecodeError
4 changes: 4 additions & 0 deletions docs/source/index.rst
Expand Up @@ -12,6 +12,10 @@ JSON Separated Values (jsv)

index

Installation
============

``pip install jsv``

Motivation
==========
Expand Down
2 changes: 1 addition & 1 deletion jsv/__init__.py
Expand Up @@ -50,7 +50,7 @@
@t2 [{2},{null}]
"""

from .template import JSVTemplate, JSVRecordDecodeError, JSVTemplateDecodeError, JSVRecordEncodeError
from .template import JSVTemplate, JSVRecordDecodeError, JSVTemplateDecodeError
from .template_io import JSVCollection, JSVReader, JSVWriter
from .__version__ import __description__, __url__, __version__, __commit_hash__, __author__, __author_email__
from .__version__ import __license__, __copyright__
10 changes: 3 additions & 7 deletions jsv/template.py
Expand Up @@ -4,10 +4,6 @@
from re import compile


class JSVRecordEncodeError(ValueError):
"""An error occurred while encoding an object into a jsv record."""


class JSVDecodeError(ValueError):

def __init__(self, msg, pos):
Expand Down Expand Up @@ -74,7 +70,7 @@ def encode(self, obj):
Args:
obj (json-compatible object): obj must also conform to the key structure of the Template, otherwise
a :class:`.JSVRecordEncodeError` will be raised.
an error will be raised.
"""
c = self._key_tree

Expand All @@ -89,8 +85,8 @@ def decode(self, s):
"""Decode a jsv string into a json-compatible object
Args:
s (str): s represents a json object that has been encoded with the given template. If it does not
conform to the template, or is unparsable as jsv, a :class:`.JSVRecordDecodeError` will be raised.
s (str): s represents a json object that has been encoded with the given template. If it does not conform
to the template, or is not parsable as jsv, a :class:`.JSVRecordDecodeError` will be raised.
"""
c = self._key_tree
if isinstance(s, str):
Expand Down
3 changes: 1 addition & 2 deletions jsv/template_io.py
Expand Up @@ -408,9 +408,8 @@ class JSVReader(JSVCollection):
Args:
record_file (filepath or :class:`io.TextIOBase`): Either a file path, or a file pointer from which records
should be read. Templates, if present, will also be read.
template_dict (dict): Dictionary of templates. See :class:`JSVCollection`.
template_file (filepath or :class:`io.TextIOBase`): Either a file path, or a file pointer to which templates
should be written. if present, templates and records will be written to different files. By convention,
should be written. If present, templates and records will be written to different files. By convention,
records should use the file extension ``.jsvr`` and templates should use file extension ``.jsvt``.
"""
Expand Down
2 changes: 2 additions & 0 deletions setup.cfg
@@ -0,0 +1,2 @@
[tool:pytest]
testpaths = test
3 changes: 3 additions & 0 deletions setup.py
Expand Up @@ -19,6 +19,9 @@
include_package_data=False,
zip_safe=True,
python_requires=">=3.4",
tests_require=[
'pytest'
],
license=about['__license__'],
data_files=[("", ["LICENSE"])],
classifiers=[
Expand Down

0 comments on commit 5ea8eca

Please sign in to comment.