Allow users to specify jq options

[sckott]

see the flags under https://stedolan.github.io/jq/manual/#Invokingjq section - @richfitz do you know if this is possible?

Some of them I think only make sense on the command line outside of R, but I think some could be used, e.g., --sort-keys

[smbache]

They could be any named arguments in the ....

[sckott]

hadn't tried it yet, just tried now, nothing works yet, an eg?

[smbache]

Why would it work yet? :)

[sckott]

im confused

[smbache]

Me too :)

My point was only the interface; that ... in the jq call, can be named and unnamed. The unnamed are used for the query itself, and the named should be passed on as options,

e.g.

jso %>% jq(".key .subkey", sort_keys = TRUE)

I guess the =TRUE part is a bit redundant here, alternatively something like

jso %>% jq(".key .subkey", .flags = list("--sort-keys"))

[smbache]

I guess you need to actually allow for the flasgs to passed along to the actual jq call under the hood

[smbache]

I can make the changes to the jq functions, provided you make it possible to pass them on.

[sckott]

Was confused as I thought you were suggesting it should work now. But I figured it shouldn't.

Right, we need to allow flags to be used. I imagine there's a way to hook into options via the C API, any thoughts @richfitz

We could use a separate function like:

json %>% jq(".key .subkey") %>% flags("--sort-keys")

I don't think this needs to happen before cran push, but perhaps it does if it means a change to how jq() works

[smbache]

Right. I guess the dsl could use a flags function. The low-level jq call could pass it as a named argument. In both cases it wouldn't be a hassle to introduce later, as adding flags after ... would not break anything (removing it again, would ;)).

So, e.g.

(jso
 %>% something
 %>% something_else
 %>% flags("--sort-keys")
)

# or
jso %>% jq("something something_else", flags = list("--sort-keys"))

I think it might get messy if we need to have low-level jq fiddled into the auto-execute story.

[sckott]

I vote for a separate function. Let's wait to see what rich says about hooking into the C API

[smbache]

Will need a bit of adjustments, I think, not that's impossible; but we need to assess whether it's worth it.. also it's not as low-level-ish ;)

[richfitz]

Sure, I'll have a look - perhaps this week.

Looks like sort-keys becomes the SORTED_OUTPUT option becomes JV_PRINT_SORTED and that's what we want to target. The other options in there are PRINT_PRETTY, PRINT_ASCII and PRINT_COLOUR (woohoo English spelling).

JV_PRINT_SORTED is consumed by jv_dump_term, passed in by a flags argument, which is passed from jv_dump_string which we use!

[richfitz]

(bear in mind though that the only connection between the command line interface and libjq is the implementation details in jq's main.c)

[richfitz]

OK, so that's done at the low level:

str <- "{\"z\": 7, \"b\": 10}"
message(jqr(str2, ".", jq_flags(sorted=TRUE,pretty=TRUE,color=TRUE)))
# {
#   "b": 10,
#   "z": 7
# }

in colour on mac terminal (not Rstudio until they get ANSI colouring). I'm not sure how you want to mix that in with the DSL stuff; perhaps it goes in query_from_dots, perhaps it goes through things like do? It's late here so I don't want to break it!

[sckott]

Thanks @richfitz

[sckott]

@richfitz Any thoughts on how to update flags methods to implement the remainder of the jq flags? I'm not quite sure from looking at main.c how to do the remainder of the flags, main.c has https://github.com/ropensci/jqr/blob/master/src/jq/main.c#L111-L127 but those are a subset of all the flags. maybe some flags we can't use in R

[richfitz]

Those flags are used only within src/jq/main.c and are not part of the libjq interface I think. We don't compile anything from that file into the library

[sckott]

Ah - how were you able to do the 4 flags you implemented then?

[richfitz]

Those are (I think) just flags to jv_dump_string, coming through here:

jqr/src/jq/jv.h

Lines 188 to 199 in e92060a

	enum jv_print_flags {
	JV_PRINT_PRETTY = 1,
	JV_PRINT_ASCII = 2,
	JV_PRINT_COLOR = 4,
	JV_PRINT_SORTED = 8,
	JV_PRINT_INVALID = 16,
	JV_PRINT_REFCOUNT = 32,
	JV_PRINT_TAB = 64,
	JV_PRINT_ISATTY = 128,
	JV_PRINT_SPACE0 = 256,
	JV_PRINT_SPACE1 = 512,
	JV_PRINT_SPACE2 = 1024,

[sckott]

thanks , hmm, doesn't seem like all the flags in the docs are in that file

[richfitz]

I think different entry points in jq have different sets of flags. Or I'm confused about what part of the docs you are seeing different sets of flags, perhaps 🎏 🇨🇦

[sckott]

the flags listed here https://stedolan.github.io/jq/manual/#Invokingjq

[richfitz]

yeah, I think that those are all doing things in src/jq/main.c and not part of the library; we'd have to reimplement that in a library style to do that in the package

[sckott]

okay, blegh

[mmuurr]

An example of an option that'd be nice is --slurp, this would make merging arrays a bit easier, like so:

x <- c('[1,2,3]','[4,5,6]')
## want: [1,2,3,4,5,6]
jq(x, 'flatten', .options = list('--slurp'))

Is there another easy way to combine (an arbitrarily-long collection of) arrays without the slurp option?

[sckott]

agree, --slurp would be nice. I don't know of another way to combine two arrays.

[sckott]

thanks for the upvote on this, will try to get it sorted

[mmuurr]

The easiest way (that I've come up with) to mimic the --slurp behavior if you have a vector of JSON array strings, is something like:

jq_slurp <- function(x) sprintf("[%s]", paste0(x, collapse = ","))
x <- c('[1,2,3]','[4,5,6]')
x %>% jq_slurp %>% jq('flatten')

This is a language-specific (i.e. R-specific) solution, however, and one of the advantages of jq bindings is the common filter string language shared across different platforms, languages, etc.

Though worth mentioning that I think jqr is already great and has drastically simplified my parsing/manipulation of JSON documents, so thanks!

[sckott]

right, there is that solution, glad at least something works. Glad jqr is already helping you out! hopefully we'll get this sorted soon