jqwim -- jq what I mean
A high-level command-line tool for filtering json data. Simpler but less powerful than
pip install git+https://github.com/talwrii/jqwim#egg=jqwim
Examples / Cheat sheet
# Search for entries whose field value matches a regular expression) jsonsource | jqwim --field-name regular-expression # Show the command line flags we can use to filter daya jsonsource | jqwim --help # Show general help jqwim --help < /dev/null
usage: make-readme.py [--help] [--or] [--spec-file SPEC_FILE] [--args-file ARGS_FILE] A limited alternative to jq with a simpler interface.The command line options depend upon your data source (or --spec-file). Provide sample data when running --help to see source specific options. optional arguments: --help Show help output --or Read specification of json from this file rather than guessing. If this file does not exist create it. This protects one against format changes causing crashes. --spec-file SPEC_FILE Read specification of json from this file rather than guessing. If this file does not exist create it. This protects one against format changes causing crashes. --args-file ARGS_FILE In addition to the command line arguments, read arguments from this this file. Dynamically update filtering as this file is updated
WARNING: Non-homogenous data
The approach used here may be a little surprising. For consistency, unix-style command line flags are used for filtering. Now new language needs to be learned. For convenience and to prevent typing errors, the acceptable flags are derived from the first record. This, however, can cause unexpected behaviour when records have different keys, or suddenly change.
For settings where strict correctness is important, for example unsupervised running in automated scripts (like cronjobs) the
--spec-file is provided.
This options specifies a json file that contains a list of acceptable keys.
When use the first time this file is built from the first key.
jq is a highly recommeded tool. It provides a domain specific language for filtering and transforming JSON. For easy use cases it can be rather verbose, however.
recordstream is a similar tool to jq. For both learning and turning-completeness (I imagine), it uses perl fragments for filtering. This has some disadvantages.
There are a number of similar young projects on github. None of these fit my use cases. Understandably, lots of these tools are "code dumps" without documentation which can make quickly working out what they do difficult. There may be other tools that provide very similar functionaly. Some examples include jsongrep, jsongrep, jqwim, jku.
Dynamic update of filters
--args-file (analogous to the
-f option for grep) allows you to load your filters from a file.
This file contains a list of command line flags.
In addition, these options will be reloaded if this file changes.
This is useful if you need to change your filters regularly.
Care should be taken to avoid sync-update race conditions however.