New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review system commands #73

Open
odino opened this Issue Dec 23, 2018 · 8 comments

Comments

Projects
None yet
3 participants
@odino
Copy link
Collaborator

odino commented Dec 23, 2018

Currently, we're limited to bash, so for example win is out of the picture. ABS executes programs with bash -c which can definitely be improved.

@odino odino added the enhancement label Dec 23, 2018

@odino odino added this to the cauldron milestone Dec 23, 2018

@nicerobot

This comment has been minimized.

Copy link
Contributor

nicerobot commented Jan 20, 2019

@odino

I wonder if you'd be open to a broader, general discussion on the language design, syntax and semantics, regarding calls and interoperability with external executables?

The $() isn't terrible but I think it can be improved while not giving the appearance of a sort of escape hatch.

Note that I'm still just 🤔 noodling this but I've been envisioning some sort of syntax that makes calling commands look and feel like standard function calls. But there's more to it than just that.

But first, just as an example, instead of

res = $(curl -s 'https://api.ipify.org?format=json');

How about adding a keyword like:

commands curl openssl tail cat

which internally just creates functions ( if each was discovered in the PATH )

func curl(args... string) { return $(curl ...args) }`
func openssl(args... string) { return $(openssl ...args) }`
func tail(args... string) { return $(tail ...args) }`
func cat(args... string) { return $(cat ...args) }`

(the $() syntax is just the effect, not intended to imply that is the implementation).

Also, it might not be technically required to provide a command keyword. You could also just consider any function name that hasn't been defined in the script(s) being executed to be external and look on the PATH.

At any rate, they can be used like

curl("-s", "https://api.ipify.org?format=json")

or, of course, if the arguments are already in an array

curl(args) // auto-unpacks as "args..." when func is varargs but parameter is slice

and maybe some sugar can be added to create arrays out of space-separated, unquoted strings, a la, bash arrays

curl(( -s https://api.ipify.org?format=json ))

The value here is that it makes external commands not seem like second-class citizens.
Additionally, i think the bash passthrough can be avoided.

But ... another reason I'm thinking along these lines is to support elegant "pipes" and "redirects" like:

cat(( a.log b.log c.log ))
  -> filter(stuff)
  -> openssl(( sha256 ))
  -> file('stuff.sha256')

Or how about something like:

tail(( -F some.log ))
  -> filter(someLines)            // grep/awk ...
  -> map(importantParts)          // perl -p -e ...
  -> each(file('important.log')) // > important.log

The key here is that all functions can be provided and expose input and output file descriptors (channels?).

And file is of special interest. It could return some sort of file-descriptor so that other things could be done like assigning to stdout and stderr which can support the redirection mentioned in #107:

savout, saverr = abs.stdout, abs.stderr
abs.stdout = file('output.log')
abs.stderr = abs.stdout

Personally, I believe this point is really crucial to any command-line interpreter. i.e. elegant handling of I/O and having it exposed literally everywhere by the core of the interpreter.

Note, again, that i'm not proposing an actual implementation. This was just off the top of my head as an example of what I think is needed and how it might be possible to implement it. I do believe the vast majority of the design of the language should focus on getting this right so that pipelines are at least as clean and easy as bash. But i think there's significant opportunity to be immensely more powerful.

Thoughts?

@odino

This comment has been minimized.

Copy link
Collaborator Author

odino commented Jan 27, 2019

@nicerobot btw I'm thinking of using backticks instead of ${}. It would solve some silly escaping issue while staying true to a familiar bash syntax. What do you think?

I've been also thinking of the pipes you mentioned. We could make it so that functions like map can accept both a fn as well as a command, and trigger the command on each element of the list. The downside is that you then trigger n commands which might not be what you want / think is happening under the good.

@nicerobot

This comment has been minimized.

Copy link
Contributor

nicerobot commented Jan 28, 2019

@odino Are you meaning `cmd` instead of $(cmd)?

I'm personally not fond of the backticks in bash scripts. Mostly for legibility but also because I like most of the Google Shell Style Guide.

My main argument against backticks and $() is that it makes it feel like it's not really a shell scripting language. $() is really more of an interpolation syntax. I think abs has an opportunity to abandon that entirely. e.g. A very common command interpolation is:

SCRIPT_DIRECTORY="$(cd -P "$(dirname "${0}")" >/dev/null; pwd)"

(Ignore the fact that this could be provided by abs natively, easily. This type of interpolation is all too common.)

The main aspects of that is what amounts to another script execution context with two sub-shells, one nested within the other. This is why i think it's prudent to think through this and come up with an elegant solution that allows for commands to feel native to the language.

For example, imagine a new type of function that can be defined with sub (for sub-shell), instead of func.

sub script_directory() {
  cd(dirname(abs.self))
  return pwd().stdout
}

I suspect abs does not need command interpolations. If commands are functions and functions support piping/file-descriptors, something like the above avoids the need for command interpolations.

re: pipes

It is the nature of pipes that all the commands in a pipeline will be a separate process (in bash). In abs, they'd likely be goroutines (each potentially backed by a process) but I'd think that should just be documented clearly when explaining abs pipes.

@odino

This comment has been minimized.

Copy link
Collaborator Author

odino commented Jan 28, 2019

I'm a bit on the fence on this, mainly because I prefer to have a terse language than end up like python -- which has good command support with minimal syntax overhead (import). I feel I'd find myself more comfortable writing:

res = `curl -s http://api.domain.com/user/me`.json()

than

res = ${curl -s http://api.domain.com/user/me`}
res = res.json()

or

use curl
res = curl -s http://api.domain.com/user/me
res = res.json()

How would you propose we go about it?

@odino

This comment has been minimized.

Copy link
Collaborator Author

odino commented Jan 28, 2019

Another example:

commands curl
res = curl("-s", "http://api.domain.com/user/me").json()

I feel this is a stretch / a bit weird :)

@nicerobot

This comment has been minimized.

Copy link
Contributor

nicerobot commented Jan 28, 2019

If you come up with an array notation, e.g.

args = ( -s http://api.domain.com/user/me )

then commands can just be

res = curl(args).json()

or as literal

res = curl(( -s http://api.domain.com/user/me )).json()

or even consider every open paren to be a whitespace-separated, string-array literal:

res = curl(-s http://api.domain.com/user/me).json()

I prefer to have a terse language than end up like python

I suppose this is where I was thinking differently. For a scripting language, I'm looking for very tight integration with the rest of the system commands with at least decent support for programming constructs. This is where bash (and others) shine. And virtually every major programming language has support to spawn/fork commands. But shells do it natively without any interference. My thinking is to imagine abs as my main shell. I'd want an interactive shell to be as nonintrusive as all the others and the syntax to be the same used to write scripts in for the shell. And I think leveraging Go's syntax is a huge win. This was why I'd like every command as a function, so I can just call anything like

foo(pass a bunch of parameters).do().awesome(things).with().the(response)

e.g.

curl(-s http://api.domain.com/user/me).json()

(Interactively, it could also support a feature like the Scala shell where every unassigned value/response is assigned to a temporary variable.)

I've written wrappers for Go to execute commands and there are utilities that will compile .go "scripts" on-demand to give them the appearance of being a scripting language. Where that falls apart is that it's not the same as my interactive shell syntax and managing the i/o (pipes, redirects, file descriptors, ...) is pretty manual/cumbersome compared to the shell. That is the primary reason for my proposal. A Go-type syntax with shell like capabilities is an awesome idea. I'd just personally want the shell capabilities to be a key driver.

@odino

This comment has been minimized.

Copy link
Collaborator Author

odino commented Jan 29, 2019

I think the main issue is that I never thought of ABS as a shell, but rather a scripting language -- though I see where you're going and I'm not specifically against it.

But as for the backticks are concerned I think it's fine to include them. Look at bash:

1. x=$(ls -la) --> implemented in abs
2. x=`ls -la` --> implemented if we include backticks
3. ls -la --> paradise, would be great to have this in abs as well

I'm not against #3, I just think it'sll be easy enough to get to 2 that I'd probably like to have it included. If we ever implement #3 then we could probably consider ABS "done" (though we would also need to implement things like history, ctrl+r etc), but I'm hopeful we could get there.

@tfga

This comment has been minimized.

Copy link

tfga commented Feb 11, 2019

The problem with backquotes is that they can't be nested.

(Acctually, in bash there is a way to do this. But it's not pretty.

Command substitutions may be nested. To nest when using the backquoted form, escape the inner backticks with backslashes.

)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment