deno eval

[ry]

Add new subcommand for evaluating on the command line

> deno eval "console.log('hello')"

Would be cool to have some examples of using Deno.stdin along with this feature in the manual.md to do unix-style line processing. Here's an imaginary command which prints all of the users on the system (note that Deno.lines is not implemented)

> cat /etc/passwd |  deno eval " \
  for await (let line of Deno.lines(Deno.stdin) { \
    console.log(line.split(':')[0]) }"

[bartlomieju]

I'll take a stab at it 🚀

[rsp]

@ry I've just watched your latest talk where you say that you'd like Deno to be your new Perl and I realized that it was basically the reason why I wrote my comments to #1038 and #929, and since one of my use cases for Perl is unix-style line processing, I recently put some thought into how the set of command-line switches of my dreams would look like, especially after my recent discussion with @bartlomieju about permission system and how it could be made more granular.

If it is not compatible with your current vision then I think I'll create some wrapper with those characteristics to use in shell scripts and command-line filters but I'd like to share my thoughts and hear if it is at all compatible with the plans for Deno command line usage.

(First of all it's interesting to see more of a git-style command-line commands like deno eval instead of deno -e like in Node, Perl, Ruby etc. but it seems that it would mean that we couldn't have files with no extensions (e.g. for command-line utils) because a script named eval (or other names if there are more commands like that) with a shebang line of #!/bin/deno would be executed as /bin/deno eval conflicting with the new command. My idea below is mapped more to traditional unix-style switches.)

All single-letter upper-case switches are shortcuts for permissions:

-R, --allow-read    Allow file system read access
-W, --allow-write   Allow file system write access
-N, --allow-net     Allow network access
-E, --allow-env     Allow environment access
-X, --allow-run     Allow running subprocesses
-A, --allow-all     Allow all permissions
# additionally:
-S, --allow-signal  Allow sending signals to processes (other than own child processes)

They can be combined as: deno -RNX script.ts and (in the future) they can take arguments:

deno -R:/etc/server -R:/var/www server.ts
deno --allow-read:/etc/server --allow-read:/var/www server.ts
# or:
deno -R=/etc/server -R=/var/www server.ts
deno --allow-read=/etc/server --allow-read=/var/www server.ts

i.e. it's a read access to specific file or a sub-tree if it's a directory.
(I'm not sure if : or = is better here. = is more visible but feels strange for multiple switches of the same kind.)

The --allow-net is for outbound connections (with optional proto/ip/host/port with wildcards or subnet masks, like. 192.168.1.* or 192.168.1.1/24 or something like that) and a separate switch is for listening, e.g.:

-L, --allow-listen Allow listening (optionally only on a specific port and/or interface)

All single-letter switches not related to permissions are lowercase, from which I think most important for use in command-line unix-style filters would be:

-e, --eval     Evaluate a line of code
-n, --???      Run the code (e.g. the `-e` line) for every line of input(*)
-p, --???      Run the code (e.g. the `-e` line) for every line of input(*) and print the result
-a, --???      Use autosplit for `-n` and `-p`

plus some switches for input/output separators and line ending processing.

(*) "every line of input" would ideally mean either stdin if there are no arguments, or reading all files if they are specified as command line arguments, like -p, -n and while (<>) in Perl, but I'd be happy if it was just the stdin because we can use cat to read multiple files with no need to give any access to those files to the Deno program.

Your recent idea with deno eval seems to be going in a different direction but I'd like to know what people think about the above, not necessarily for the deno itself (though it would be most convenient) but maybe for a separate tool, something like my filt module on npm but more advanced.

[bartlomieju]

To extend @rsp's example of CLI args: we talked about "permission maps" which essentially could be a JSON file similar in structure to import-map. It would allow to granularly specify what is allowed for package and its imports.
Example:

deno --permission-map=perms.json script.ts

// perms.json
{
    "https://deno.land/": {
       // standard modules have full permissions
        "read": true,
        "write": true,
        "net": true,
        "run": true,
        "env": true
    },
    "https://deno.land/x/postgres": {
        // allow `dial` but only for these adresses
        "net": [
            "0.0.0.0",
            "127.0.0.1",
            "192.168.1.1/24"
        ],
        "env": true
    },
    "<some_other_package>": {
        "write": [
            "/var/www"
        ],
        "read": [
            "/var/www",
            "/var/server"
        ]
    },
    // catch-all for unknown packages, deny all
    "*": {
        "read": false,
        "write": false,
        "net": false,
        "run": false,
        "env": false
    }
}

This solution requires that permissions are "contextful" - during op we'd need to know which module issue this call and resolve it against the map. That's probably a good place to move discussion to dedicated issue for that.

[ry]

@rsp You bring up a couple of different ideas - I will try to address them individually

1. "deno eval" means you cannot execute a javascript file named "eval". That's a good point and I hadn't considered. Maybe it's not such a big deal since there's only a finite number of subcommands. As long as you avoid "fmt" "eval" "types" and a few others it should still be possible to use extensionless scripts...

2. The case against only using traditional unix flags (and no subcommands) is this: not all flags are appropriate in every combination. For example, if "deno fmt" was instead "deno --fmt" then it wouldn't make sense to also combine it with "--info".

3. Regarding the whitelisted directories (and network connections): this keeps coming up, so I think we'll have to implement this.

4. Regarding single letter switches. I don't really know the final shape of the CLI flags so in general I'm trying to be conservative in adding flags (especially abbreviated ones). I would like to postpone committing to single letter switches for now.

5. filt looks amazing. I would love for deno to support such succinct command line examples.

@bartlomieju While that seems very useful, I don't want to unilaterally introduce new file formats. (That's how we got package.json.) Maybe there is something in the web package standards that is applicable. In any case, I think this is something we can punt on for a few months.

[hayd]

The other issue with the json is does write access (to that file) imply permissions could be increased on subsequent executions. So would you need to write protect it somehow (I guess you could have a blacklist of write access as well as a whitelist).

Is there a standard within CSP?

Kevin had a good solution for 1. having run as (the default) subcommand, that way you can use deno run eval if you need to disambiguate (run a file named eval). Alternatively perhaps you'd use ./eval ?

[rsp]

@ry

1. as for the filename not being able to be namedeval, maybe if we have an (optional) command run then we could use it in such cases:

$ deno foo bar
# runs ./foo with argument "bar"

$ deno eval bar
# runs "bar" eval-ed as JS (or TS? good question: do we need two commands? or `eval --ts`?)

$ deno run eval bar
# runs ./eval with argument "bar"

2. The more I think about the idea of dash-less commands, the more I like it. Seeing how well it works with git, it could give us context to know which flags can be used together plus a lot of space for new features in the future. If we have run mandatory (maybe called x to be shorter?) then we won't ever risk breaking existing scripts by adding new commands (it seemed strange to me at first when I thought about it but not that much, really). It would give space for things like deno verify x.ts or deno compile x.ts and other interesting things other than just running scripts come to mind.

3. Great. One more thing to that: currently --allow-run is carte blanche. A script with unrestricted power to run scripts can run itself with --allow-all. Have you thought about a permission (plus a user-space command) to run only other Deno programs using the same interpreter? If we have a command similar to Deno.run() (or an option to Deno.run()) but only for other Deno programs to run them with a subset of parent's permissions, e.g.:

$ cat a.ts
Deno.denoRun('b.ts', { allowWrite: true, allowRead: true });

$ deno --alow-read a.ts
# either runs child with just `--alow-read` or throws (not sure what's better here)

This would allow Deno programs run other Deno programs without ruining the permission system by having them run with general --allow-run. Of course a general --allow-run will be needed as well, ideally with a whitelist like --allow-run=/bin/ls but the Deno permission system is too good to throw it away when running child Deno processes.

4. True, maybe at this point we shouldn't have any single letter switches to see later what is used most. My idea was to reserve upper-case single letters for permissions because we would have many of them and it would be important to not make a mistake but we don't have to assign them yet.

5. Thanks, good to hear, I'll write something similar for Deno as a proof of concept and we'll see what can be supported by deno itself and what would be better as a separate tool.

[rsp]

@hayd when I was talking with @bartlomieju about the permissions map, what I had in mind (maybe I didn't express it explicitly) was not something that grants privileges, but rather something that restricts them, and the fact of it being in a JSON file or not would be something irrelevant.

Maybe I'll use an example:

A file ./script.ts starts with something like:

const perms = {
    "./lib.ts": {
        "read": true
    },
    "https://deno.land/x/postgres": {
        "net": true,
        "env": [ "POSTGRES_URI" ]
    }
};
Deno.usePerms(perms);
import { x } from './lib.ts';
import { y } from 'https://deno.land/x/postgres';

(the perms object can be read from a JSON file, or imported from a ./perms.ts file or whatever)

Now, the script.ts itself must still be run as: deno -RNE script.ts (or deno run -RNE script.ts, see my comment above) to have those permissions to begin with (another way I see would be to have a declaration of wanted permissions in a program that the user would have to interactively agree to (maybe that decision could be remembered, or maybe another switch for that - that's another matter of how the original permissions could be granted to the main program) but the perms object in my example above would be to give a subset of the main program's permissions to the dependencies.

I think few different things should be separated here:

1. granting the privileges to the main program itself
2. restricting those privileges to the dependencies
3. dropping the privileges in the main program

(1) is what I discussed in my two comments above in this issue and in other comments linked there, (2) is what I would imagine would be the purpose of an object/JSON as written by Bartek, and (3) would be something that I think needs some attention too. We currently have Deno.revokePermission() and what I'm thinking is if that is enough or maybe some other mechanism would be useful to be able to audit large systems easily.

Ideally it would be compatible with mechanisms like OpenBSD's pledge() (the new one) and unveil():

• https://man.openbsd.org/pledge.2
• https://man.openbsd.org/unveil.2

I think that having such a strong focus on security as Deno has, it would be useful to work nicely with operating systems that share that sentiment. When I wrote issue #378 on capability-based security model I was thinking more in the style of GNOSIS, KeyKOS, EROS, CapROS and Coyotos but maybe we should focus more OpenBSD that is often used for applications where security is important and that has some interesting mechanisms with which it would be nice to be compatible. I'm sure that Deno will run on OpenBSD in the future and it would be interesting to have the first scripting language runtime to support native security features of OpenBSD.