Easy support for top-level functions

[copumpkin]

For a variety of reasons (composability foremost), I quite like writing top-level functions in my jsonnet files, rather than using std.extVar. Unfortunately, to invoke them from the command line, I need to surround those top-level functions with a snippet that imports my file and passes in the correct parameters.

It would be nice if there were a simple way to take jsonnet files with top-level functions and invoke them on the command line, with similar options to -E, -V, and --code-var. Or even just -V and --code-var, since the other two can be easily approximated.

[copumpkin]

The simplest way to do this in a general way:

jsonnet -e foo.jsonnet --exec "function (expr) expr(5, 8)"

The semantics being that the --exec parameter is a function that gets passed the expression loaded from the file you specified on the command line.

Today's jsonnet -e foo.jsonnet would be equivalent to jsonnet -e foo.jsonnet --exec "function (x) x".

Command-line CPS!

[sparkprime]

Agree that std.extVar is global state and a blunt instrument. I have thought the same thing -- that we might allow people to write configs that are functions, and then the bound variables would not be visible throughout.

--exec already exists as an alias of -e

Why not have some --local-var command that binds a variable implicitly at the top level, like std is bound implicitly?

[copumpkin]

My main motivation is for jsonnet files that I want to be able to invoke in isolation and import from other files. If I make a top-level function, I can pass in what I want from the command line, and also pass in what I want from another file.

With extVar and your --local-var solution, I can't.

There is a bit of precedent for this discussion, FWIW. In nixpkgs, pretty much every "package" is a top-level function from its dependencies (and other flags) to a "buildable thing". We then import that function and pass in its dependencies, basically hooking everything up together a bit like a dependency injection framework might do.

However, people (understandably) get a bit sick of writing some parameter names three times: once in the package call-site, once in the list of parameters for the package, and again wherever it's used inside the package expression. This led to an experimental "import with scope" primitive being added to the language called scopedImport, which basically behaves like import except it lets you pass along a scope while importing.

We haven't really adopted it much because people feel uneasy about magic variables appearing out of nowhere, without being declared as explicit dependencies somewhere prominent. But there are still proponents of it, and I think people are still experimenting with sensible ways to use it.

Basically, for composability's sake, if you do the --local-var solution, I'd really like some sort of accompanying primop that has the same effect in code. In the absence of a REPL and debugger, I quite enjoy being able to take individual jsonnet files that I would normally be importing, and evaluate them independently (I wrote a simple python wrapper that does that right now) to see how they behave.

Does that make sense? Sorry for the wall of text.

[sparkprime]

@oconnorr it's the n word again

There is a really old issue to add a REPL. I'm not sure how that would work / how to implement it, but it sounds like a good idea.

If there was a REPL, would you still want this?

[copumpkin]

If the REPL gave me the equivalent of scopedImport with your --local-var option, I'd be fine without it. Hell, I'm fine with it today (my python shim is 5 lines), but it would be nice to not need anything else.

[sparkprime]

Can you paste your python shim (just curious)

[copumpkin]

Sure (turns out I lied about the lines)!

import _jsonnet
import json

def import_jsonnet(path, **kwargs):
    jsonnet_str = '''
        local args = %s;
        local fun = import "%s";
        fun(args)
    ''' % (json.dumps(kwargs), path)
    json_str = _jsonnet.evaluate_snippet("(dyn)", jsonnet_str)
    return json.loads(json_str)

[sparkprime]

Cute :)

Yeah the fact that files designed for --local-var cannot be otherwise imported is pretty weird. That's probably not a good direction to go in.

[sparkprime]

I think the commandline interface gets a lot nicer if you only allow one arg, like in that python file. People are free to pass in a JSON object.

[copumpkin]

Yeah, definitely. That was actually part of the motivation for my #119, since I'm basically doing that all over the place because of this snippet.

[copumpkin]

(although obviously if we had a splat operator we wouldn't have to choose)

[sparkprime]

@mikedanese would this help with your k-on-k stuff?

[mikedanese]

If I understand what is proposed, I think that the top level function would be useful for implementing the unit testing framework, yaml stream, a k8s object templater, etc... It makes the (input -> function -> output) flow a little bit cleaner. Is this generalizable to a custom manifestation/finalizers?

[sparkprime]

It's sorta related to that but a bit different. With the manifest stuff, the function is the manifester, which you add onto the config. With the original proposal here, the config becomes a function, and the extra code on the side calls that function.

More generally, I was wondering how you'd feel about replacing std.extVar with some top level function and threading through the variable? Would that improve your scripts?

[sparkprime]

Fixed by #164