-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
partially compact ("collapse"?) JSON output #643
Comments
Would this alter the structure of the JSON object outputted, or just the |
It would do the exact same thing as It reduces 31 lines of pretty-printed JSON to 7 equivalent lines. |
A jq function could do this, no doubt, though you'd have to use the raw |
I see. Well, there's obviously a crucial difference, in that What @nicowilliams proposes, while feasible, doesn't sound practical, but maybe that's just me. Could you expand on which is the use case for this feature? Is this a readability concern? What is exactly the problem that is being solved? How would you feel about a version of jq --collapse-output=80 "." something.geojson
{
"type": "Feature",
"properties": [],
"geometry": {
"type": "Polygon",
"coordinates": [
[
[0.2944921875, -0.49865941695124],
[0.2944921875, -0.48077953413874],
[0.49800732421875, -0.48077953413874],
[0.49800732421875, -0.49865941695124],
[0.2944921875, -0.49865941695124]
]
]
}
} |
bump |
@evandrix What do you propose? |
@kjell Why do you want this? Is it to help visualize some JSON? If that's all then.. just |
jq is a JSON processor, but it works mainly with parsed values, which is why we have so many issues complaining about changes in formatting of numbers, for example (and before that order of object names/keys). jq is not a very formatter: it has the kinds of options most JSON encoders have (e.g., compact vs. pretty-printed with some number of spaces for indentation), but that's it. Adding a way to specify different formatting options at different parts of a JSON value tree is... not trivial. It could be done, I suppose, but presumably the paths where different options are to be used should be expressed as jq expressions, which would have to be evaluated by... the encoder, which kinda means that either the encoder runs a jq program separate from the main one, or the encoder has to be jq-coded itself. I think it would have to be the latter. A jq-coded encoder is not infeasible, but as @slapresta says, it's not terribly practical, though mostly because one doesn't exist yet. |
I think this makes more sense as a second program.
It's not so much to visualize the JSON as to make it as human-readable as possible without deleting anything. In my example above, my brain will never understand those lat/longs no matter how they're printed. It's enough to know that it has some coordinates. But glancing at the JSON I might care about I'm thinking of it like JSON chartjunk, "a lot of ink that does not tell the viewer anything new". So minimizing it would be great, but it's not important. (And sorry for my silence, I lost track of this over the new year.) |
As an alternative, how about altering the behavior of the flag to compact below a certain depth, where the current behavior is effectively |
@nicowilliams I wish jq could be a little smarter at pretty-printing. For example, here is what underscore-cli pretty-prints:
Here is jq output:
|
While looking at kjell's post, I think having it be a filter feels the best, something with the default nature of Although, probably with what nicowilliams has said, if I'm understanding the tool correctly, filters just operate on the data, and the formatting stage is last and separate from our control. That could change possibly, it would just need some engineering. Maybe data going through the chain of filters would get 'flagged' with any specific formatting options chosen for them. In the late formatting stage, the 'global' tool flags would be used for all data segments as the iteration process executes, but the specific flags would take precedence if any were found. The example and idea from balta2ar does also sound good, just not as potentially flexible, and if it is easy enough to implement, maybe leave it open to some configuration at least, if possible. |
Assuming a In principle we could remove, or, more likely, reimplement a number of jq command-line options and behaviors by just wrapping the top-level program with jq code that manages standard I/O as expected, with the Because of that we're reluctant to add new command-line arguments where one can get the same behavior from appropriate use of existing options and jq builtins. |
@balta2ar thanks for pointing me to underscore_cli |
Re how to do and specify this, the Haskell people have long ago solved the general case using backtracking. See Hughes (1995), Wadler (1998), discussion in elixir-lang/elixir#1047. There is also, apparently, an algorithm of Oppen that makes choosing the optimal layout linear-time. Not sure if such generality is actually necessary in this rather limited case, though. |
Python $ cat wh.py
xr = [
[
1428578978,
"yt_I4Uy3Zjg50o",
"Camera Obscura - Desire Lines (4AD Session)",
2013
],
[
1428578548,
"yt_199XS8ucYiM",
"Doe Paoro - Traveling",
2015
],
[
1428578095,
"yt_TBt62fl1ZLM",
"Trance Soundtrack - Raw Umber",
2013
]
]
import pprint
pprint.pprint(xr, compact=True) Result: $ python3 wh.py
[[1428578978, 'yt_I4Uy3Zjg50o', 'Camera Obscura - Desire Lines (4AD Session)',
2013],
[1428578548, 'yt_199XS8ucYiM', 'Doe Paoro - Traveling', 2015],
[1428578095, 'yt_TBt62fl1ZLM', 'Trance Soundtrack - Raw Umber', 2013]] |
My script puts the last level of braces on one line. Feed the output of "jq ." to stdin. I think it can be extended to support the last level of brackets as well.
|
underscore (https://github.com/ddopson/underscore-cli) has a parameter that influences the degree of compactness, e.g.
and:
|
Here is what I'm using for inspecting a json log file
|
I think this would be 80% of what i need. For example, it would turn this:
Into the substantially more readable:
From reading the discussion above, perhaps this does not belong in jq, which is focused on the content of the JSON, but in some separate tool that is specifically about pretty-printing. |
was there any attempt to implement this in |
Still as relevant as ever.. And I second the idea of having the wrapping width as a parameter. |
Same need here but for me a max-printing depth parameter could also be handy. |
@yanOnGithub - I used your little JSON compactor as part of a tool I created for compacting some Elasticsearch template files (written in JSON). The tool can be found here: https://github.com/Jan-Bruun-Andersen/elastic-legacy-to-composable |
I like this idea of {
"1142": {
"end_datetime": "2017-03-14 23:59",
"lambda": { "caps": { }, "pacing": "asap" },
"start_datetime": "2016-03-15 00:00",
"targeting": {
"weekly_schedule": {
"Fri": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
"Mon": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
"Sat": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
"Sun": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
"Thu": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
"Tue": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23],
"Wed": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23]
}
}
}, or similar to https://github.com/j-brooke/FracturedJson output {
"SimpleArray": [
2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61,
67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109, 113
],
"ObjectColumnsArrayRows": {
"Katherine": ["blue" , "lightblue", "black" ],
"Logan" : ["yellow" , "blue" , "black", "red"],
"Erik" : ["red" , "purple" ],
"Jean" : ["lightgreen", "yellow" , "black" ]
},
"ArrayColumnsObjectRows": [
{ "type": "turret" , "hp": 400, "loc": {"x": 47, "y": -4}, "flags": "S" },
{ "type": "assassin" , "hp": 80, "loc": {"x": 12, "y": 6}, "flags": "Q" },
{ "type": "berserker", "hp": 150, "loc": {"x": 0, "y": 0} },
{ "type": "pittrap" , "loc": {"x": 10, "y": -14}, "flags": "S,I" }
],
"ComplexArray": [
[19, 2],
[ 3, 8],
[14, 0],
[ 9, 9],
[ 9, 9],
[ 0, 3],
[10, 1],
[ 9, 1],
[ 9, 2],
[ 6, 13],
[18, 5],
[ 4, 11],
[12, 2]
]
} |
bump |
Someone asked about this on IRC a while ago. A while even earlier, I happened to have written this #!/bin/sh --
# \
exec jq -f "$0" -r --args -- "$@"
def toprettyjson($n):
def _toprettyjson($l):
($l * $n * " " // "") as $indent |
if (isempty(scalars) | not) or IN({}, []) then
tojson
elif type == "object" then
"{\n" + (
to_entries |
map($indent + ($n * " " // "") +
"\(.key | tojson): \(.value | _toprettyjson($l + 1))"
) |
join(",\n")
) + "\n" + $indent + "}"
else # array
"[\n" + (
map($indent + ($n * " " // "") +
_toprettyjson($l + 1)) |
join(",\n")
) + "\n" + $indent + "]"
end;
_toprettyjson(0);
toprettyjson($ARGS.positional[0] | tonumber? // 4) They wanted to "compact" #!/bin/sh --
# \
exec jq -f "$0" -r --args -- "$@"
def toprettyjson($n):
def _toprettyjson($l; $path):
($l * $n * " " // "") as $indent |
if (isempty(scalars) | not) or IN({}, []) then
tojson
elif type == "object" then
if $path == [ "windows", 0, "tabs", 0 ] then
tojson
else
"{\n" + (
to_entries |
map($indent + ($n * " " // "") +
"\(.key | tojson): \(
.key as $k |
.value | _toprettyjson($l + 1; $path + [ $k ]))"
) |
join(",\n")
) + "\n" + $indent + "}"
end
else # array
"[\n" + (
map($indent + ($n * " " // "") +
_toprettyjson($l + 1; $path + [0])) |
join(",\n")
) + "\n" + $indent + "]"
end;
_toprettyjson(0; []);
toprettyjson($ARGS.positional[0] | tonumber? // 4) Example:
|
Yes, it would be very nice to have this as a native feature in jq. |
@nicowilliams You could easily implement that adapting my code as... def tocompactjson(p; $n):
[ canonicalize_path(path(p)) ] as $paths |
def _tocompactjson($l; $path):
($l * $n * " " // "") as $indent |
if IN($paths[]; $path) or (isempty(scalars)|not) or IN({}, []) then
tojson
elif type == "object" then
"{\n" + (
to_entries |
map($indent + ($n * " " // "") +
"\(.key | tojson): \(
.key as $k |
.value | _tocompactjson($l + 1; $path + [ $k ]))"
) |
join(",\n")
) + "\n" + $indent + "}"
else # array
"[\n" + (
[
range(length) as $i |
.[$i] |
$indent + ($n * " " // "") +
_tocompactjson($l + 1; $path + [$i])
] |
join(",\n")
) + "\n" + $indent + "]"
end;
_tocompactjson(0; []); if Proof of concept with idenitity $ jq -r 'def canonicalize_path(x): x; def tocompactjson(p; $n):
[ canonicalize_path(path(p)) ] as $paths |
def _tocompactjson($l; $path):
($l * $n * " " // "") as $indent |
if IN($paths[]; $path) or (isempty(scalars)|not) or IN({}, []) then
tojson
elif type == "object" then
"{\n" + (
to_entries |
map($indent + ($n * " " // "") +
"\(.key | tojson): \(
.key as $k |
.value | _tocompactjson($l + 1; $path + [ $k ]))"
) |
join(",\n")
) + "\n" + $indent + "}"
else # array
"[\n" + (
[
range(length) as $i |
.[$i] |
$indent + ($n * " " // "") +
_tocompactjson($l + 1; $path + [$i])
] |
join(",\n")
) + "\n" + $indent + "]"
end;
_tocompactjson(0; []);
tocompactjson(.foo, .baz[2]; 2)
' <<< '
{"foo":[{"1":2,"3":null},1],"baz":[{"1":2,"3":null},1,{"1":2,"3":null},3],"bar":[{"1":2,"3":null},1]}
'
{
"foo": [{"1":2,"3":null},1],
"baz": [
{
"1": 2,
"3": null
},
1,
{"1":2,"3":null},
3
],
"bar": [
{
"1": 2,
"3": null
},
1
]
} |
any updates? |
I'd like to output "mostly pretty" JSON, with overly verbose objects compacted. Something like:
Has this been discussed anywhere?
The text was updated successfully, but these errors were encountered: