An absolutely simple & terse CLI app to manipulate JSON data
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
app
src Initial commit Nov 12, 2017
test
.gitignore
LICENSE
README.md
Setup.hs
jsn.cabal
stack.yaml

README.md

jsn

jsn (pronunciation: json) - an absolutely simple & terse CLI app to manipulate JSON data. You either learn to use it in less than 10 minutes, or I failed.

Goals:

  1. As terse, simple and mnemonic as possible, so that you can learn the core functionality in 10 minutes.
  2. Once you learn it, you can instantly use it efficiently - out-of-box experience thanks to the sane defaults.
  3. The less features, the better. Unless requested features are proven to have a common application.
  4. Good at one thing - JSON data manipulation. No stuff like mathematical operations, as there are better tools for it. Prioritize performance, perfect error handling and usability with other apps over new features.

So, what is it actually good at?

  • doing quick modifications to JSON files from command line - it's great for configuration files like package.json, where all you want to do is insert/remove/change a small portion of an already big and complexed (in terms of structure) file
  • general operations on JSON data, thanks to its streaming interface - you can combine (pipe) it with other apps for unlimited possibilites - it's quite fast too

Start

Download: there are binaries available for macOS and Ubuntu. I did not test any other platform yet.

To get started, complete this short tutorial. Run jsn -h for additional help.

Learn it in 10 minutes

Firstly, let's define 2 terms which are used across this project to keep things simple:

  • member - an element of JSON array or a value of JSON object
  • key - an index of JSON array or a key of JSON object

A list of keys forms a path to access a member.

jsn has commands, options and flags. Each of them is always denoted by one letter.

Every command consumes a JSON input to perform some manipulations on it and produces a JSON output.

By default, input is read from stdin and output is written to stdout. You can also use these options to change it:

  • -f FILE to read input from a file named FILE
  • -i FILE to perform an in-place edit - read input from FILE and write output to it

Ok, let's run our first command. Consider a file called file.json:

{
  "data": {
    "numbers": [0, 1, 2]
  }
}

We modify it by inserting (i command) a new number into the list:

jsn i '.data.numbers' '3' -i file.json

file.json is now:

{
  "data": {
    "numbers": [0, 1, 2, 3]
  }
}

How about inserting an element to the beginning of the array? Well, we need to use a reversed variant of our insert command. That is right: some jsn commands have a reversed variant. It is denoted by an uppercase command name:

jsn I '.data.numbers' 'null' -i file.json

file.json is now:

{
  "data": {
    "numbers": [null, 0, 1, 2, 3]
  }
}

The syntax for almost every jsn command is more or less the same:

jsn COMMAND PATH [VALUE]

Where:

  • COMMAND is a command name (uppercase for a reversed variant)
  • PATH is a path pattern to find members. It's syntax is: .key1.key2.key3, but generally key1.key2.key3 is fine too. An empty string ('') means a root member, so just the JSON input itself. Keep in mind that 0, 1 or more paths can match a path pattern. For example, our previous file has 1 member for pattern .data.numbers, but 0 members for pattern .numbers. To match multiple paths, we can use wildcards, which are similar to glob: * means exactly one key and ** means 0 or more keys. So, given our example, we could also use path pattern .**.numbers or .*.*. Both of them would perform the same insertion.
  • VALUE, which exists only for some commands (like insert), is just a JSON value. The cool part is that, once it's missing, it will be also read from stdin. This provides us a great interface for piping jsn commands!

Let's consider a new file, primes.json:

[2, 3, 5, 7, 11]

We want to take the last 3 primes and concatenate them with our previous array of numbers. Alright, let's do it:

jsn t '' 3 -f primes.json | jsn c 'data.numbers' -i file.json

file.json is now:

{
  "data": {
    "numbers": [null, 0, 1, 2, 3, 5, 7, 11]
  }
}

How does it work?

  1. Command t (syntax: t PATH N) reads input from file primes.json and takes last 3 (N argument) elements from it - a path pattern '' means the root of JSON input.
  2. Command c (syntax: c PATH VALUE), thanks to the pipe operator, uses the previous output for a VALUE argument and concatenates it with a member at data.numbers path. The input is read from file.json and written to the same file (-i for an in-place edit).

For our last example, consider this input:

{
  "array": [
    {
      "list": [
        null,
        {
          "foo": true
        }
      ]
    }
  ]
}

How do we set the true value to false? We need a set command (s):

jsn s '.array.0.list.1.foo' 'false'

It works, but it's not great. How can we access the value in a more elegant way? Wildcards!

jsn s '.*.*.*.*.foo' 'false'

Or even just:

jsn s '.*.*.*.*.*' 'false'

Or...

jsn s '.**.foo' 'false'

Much more elegant, right?

Commands cheatsheet

name arguments action reversed variant
g PATH [-a] get all values matching path (-a - array of values) -
s PATH VALUE set member value -
n PATH VALUE new member (like s, but can create a new member) -
d PATH delete member -
k PATH keys of object -
v PATH values of object -
l PATH length of array or object -
r PATH reverse array -
t PATH N [-b IND=0] take N elements from array, begin at IND (default: 0) take elements from the beginning
p PATH [N=1] pop N (default: 1) elements from array pop elements from the beginning
i PATH VALUE insert VALUE to array insert element to the beginning
c PATH VALUE concatenate array and VALUE (VALUE must be array) concatenate VALUE and array
m PATH VALUE merge object and VALUE (VALUE must be object) overwrite existing keys
f PATH [KEYS...] filter (keep) members of a member by keys filter out (discard) members

Gotchas

No path matched our path pattern

If one or more members matches PATH pattern, the command action will be performed on every matched member - starting from the deepest ones. If PATH doesn't match anything, you will get an error (a sane default, right?). How to disable it? Use -z flag which stands for: zero paths matched allowed. So, what you will get with a -z flag enabled and 0 paths matched? Naturally, output will be the same as input.

Tolerant mode

It can happen, that some paths matched don't make sense for your command and generate an error. For example, concatenation command jsn c '*' '[1, 2, 3]' will fail for this input:

{
  "a": [0],
  "b": true
}

While member at path .a is an array, member at path .b clearly isn't. What if we would like to ignore those fails? We can use a tolerant mode with flag -t. A command jsn c '*' '[1, 2, 3]' -t will give us our desired result:

{
  "a": [0, 1, 2, 3],
  "b": true
}

Get command (g)

Every command shares the same output format. The only exception is a get command (g). If the PATH pattern matches zero or one path, then it's quite the same. What happens if it matches many members, though? The answer is: it will write multiple outputs. While it can be useful to use with other apps, it's impossible to pipe it through jsn itself - jsn always expects one input. That's why you can use an -a flag, which stands for "array output". All the outputs will be combined to one array of JSON values - which is a valid JSON value. Voila!

New command (n)

n command is quite unique. It's behaviour is simple: do what set command (s) does, but accept a case where there is no such a member yet.

Consider an input:

{
  "list": [1, 1, 2, 3]
}

So, while both jsn s '.list.0' '0' and jsn n '.list.0' '0' will produce:

{
  "list": [0, 1, 2, 3]
}

A command jsn s '.list.4' '4' will simply fail - there is no such a member. Command jsn n '.list.4' '4' will do just fine, though:

{
  "list": [0, 1, 2, 3, 4]
}

Filter command (f)

f command accepts KEYS argument to filter... members of a member. This doesn't sound like a straightforward thing, even though it really is. Suppose we have this input:

{
  "data": {
    "a": 1,
    "b": 2,
    "c": 3
  }
}

If we run jsn f '.data' 'a,b' we get in return:

{
  "data": {
    "a": 1,
    "b": 2
  }
}

And the reversed variant: jsn F '.data' 'a,b'? Couldn't be any more obvious:

{
  "data": {
    "c": 3
  }
}

You can either separate keys by a comma, or just use multiple arguments. Heck, you can even do both: jsn F '.data' 'key1,key2' 'key3' 'key4'!

"I only want the modified part"

Every jsn command will always return the modified root member of input as output. Such a consistency is great, but very often it can be counter intuitive. Consider such a file:

{
  "numbers": [1, 2, 3],
  "otherData": "foobar"
}

What's the output of the command: jsn t '.numbers' 2? It is:

{
  "numbers": [2, 3],
  "otherData": "foobar"
}

"What?! But I've literally asked to take 2 elements, nothing else!".

I see your point! To use jsn commands like this, just apply -m flag, which stands for a "modified part". This way, a command jsn t 'numbers' 2 -m would give us:

[2, 3]

It even works with wildcards matching multiple paths. Although, you need to be aware that paths are always matched starting from the deepest members and in such an order all the operations are performed. So what you eventually get is the outermost modified member.

Those boring edge cases

Sometimes, out of insanity, people will be tempted to use JSON standard to the highest extent and use the following characters for their keys:

  • empty string ('')
  • dots (.)
  • stars (*)

While it's far from being a popular practise, jsn does support it. To not collide with other features, you simply need to apply an escape operator (\). So, to access the deepest member in input like:

"**": {
  "": {
    ".\\": true
  }
}

One has to create such a path:

.\*\*..\.\\

Which is not the most readable line of code ever, but people used to regular expression will surely not mind :) .


And that's the entire jsn API. I hope that reading this didn't take you much more than the promised 10 minutes!

Pretty print

By default, every output will be pretty printed. If you want to change the whitespace characters used for indentation (default: 2 spaces), use -w SPACES option, where SPACES is the amount of spaces. If you want tabs, simply use 't' value.

If you want to disable pretty printing, just use the -u (ugly) flag.

Why is it so terse?

And why is vim so terse with its commands? Because small tasks require small amount of time spent on them. Way too often I need to perform a quick JSON modification, which turns out to be not so quick at all.

I struggle. No trailing commas allowed - oh, what a shame. I can not just omit double quotes like in JavaScript? Not so great. Oh, I need to copy and paste values here and there, because there is no simpler way for it. My frustration has led me to creating jsn.

If you really dislike this convention, simply alias everything.

Contributions

Are welcome in any form. Share your bugs, ideas, pull requests. Add tests or refactor the code. Thank you very much for your attention, you're the best ;) .

Alternatives

Almost every other tool built for JSON manipulations does so much more. jsn was built to be as dumb as it's possible, so that you can pick it up quickly. However, the following tools are great for more advanced things:

  • jq - it has pretty much everything you could think of
  • underscore-cli - a very powerful JS-like interface

TODO

Add tests. Add benchmarks to measure and improve perf.

License

MIT