Skip to content

prataprc/jsondata

Repository files navigation

Why yet another JSON package in Rust ?

Rustdoc GitPitch Build Status

This crate makes several trade-offs that are tuned for big-data and document database.

  • Support for 128-bit signed integers.
  • Deferred conversion for JSON numbers.
  • Serialization from Rust native type to JSON text.
  • De-serialization from JSON text to Rust native type.
  • CRUD operation on JSON documents, using JSON Pointer.
  • Sorted keys in property object.
  • Streaming JSON parser.
  • Support JSON5 standard.
  • Common arithmetic and logic operations.
  • Sortable JSON.

Useful links

Deferred conversion for numbers

Converting JSON numbers to Rust native type is not always desired. Especially in the context of big-data where data is stored in JSON format and we need to lookup, only, specific fields within the document.

This implementation provides deferred conversion for JSON numbers that leads to a performance improvement of upto 30%.

Caveat: If numerical text is greated than 128 characters, deferred conversion won't work, that is, text shall be parsed immediately.

CRUD operations on JSON document

Using Json Pointer it is possible to identify a specific field nested within a JSON document. For Example, with below document:

  {
    "age": 26,
    "eyeColor": "green",
    "name": "Leon Robertson",
    "gender": "male",
    "company": "AEORA",
    "phone": "+1 (835) 447-2960",
    "tags": [ "officia", "reprehenderit", "magna" ],
    "friends": [
      {
        "id": 0,
        "name": "Glenda Chan"
      }
    ]
  }
  • /age shall point to value 26.
  • /tags shall point to value [ "officia", "reprehenderit", "magna" ].
  • /tags/0 shall point to value "officia".
  • /friends shall point to value [{"id": 0, "name": "Glenda Chan"}].
  • /friends/name shall point to value "Glenda Chan".

List of operations

  • Get a field nested within a JSON document using JSON Pointer.
  • Set a field nested within a JSON document.
  • Delete a field nested within a JSON document.
  • Append string or array field withing a JSON document.

JSON5

  • Object keys may be an ECMAScript 5.1 IdentifierName.
  • Objects may have a single trailing comma.
  • Arrays may have a single trailing comma.
  • Strings may be single quoted.
  • Strings may span multiple lines by escaping new line characters.
  • Strings may include character escapes.
  • Numbers may be hexadecimal.
  • Numbers may have a leading or trailing decimal point.
  • Numbers may be IEEE 754 positive infinity, negative infinity, and NaN.
  • Numbers may begin with an explicit plus sign.
  • Single and multi-line comments are allowed.
  • Additional white space characters are allowed.

Track this feature.

Sortable JSON

  • Null type shall sort before all other types.
  • Boolean type shall sort after Null type.
  • Number type shall sort after Boolean type.
    • f64 values that are <= -2^127 will sort before all i128 integers.
    • f64 values that are >= 2^127-1 will sort after all i128 integers.
    • NaN, Not a Number, values shall sort after all i128 integers
    • -Infinity shall sort before all numbers.
    • +Infinity shall sort after all numbers.
    • NaN shall sort after +Infinity.
  • String type shall sort after Number type.
  • Array type shall sort after String type.
  • Object type shall sort after Array type.
    • All (key,value) pairs within the object shall be presorted based on the key.
    • When comparing two objects, comparison shall start from first key and proceed to the last key.
    • If two keys are equal at a given position within the objects, then its corresponding values shall be compared.
    • When one object is a subset of another object, as in, if one object contain all the (key,value) properties that the other object has then it shall sort before the other object.

Useful links

Operations on JSON documents

  • Arithmetic operations, ADD, SUB, MUL, DIV, REM, NEG.
  • Bitwise operations, SHL, SHR, BITAND, BITOR, BITXOR.
  • Logical operations, NOT, AND, OR.
  • Index operations.
  • Range operations.

Detailed description can be found here.

Contribution

  • Simple workflow. Fork - Modify - Pull request.
  • Before creating a PR,
    • Run make build to confirm all versions of build is passing with 0 warnings and 0 errors.
    • Run check.sh with 0 warnings, 0 errors and all testcases passing.
    • Run perf.sh with 0 warnings, 0 errors and all testcases passing.
    • Install and run cargo spellcheck to remove common spelling mistakes.
  • Developer certificate of origin is preferred.