[|\] Merge branch 'dashes' into develop.

Makefile

@@ -11,6 +11,11 @@ docs:
 install: tests
 	python3 setup.py install --prefix=$(INSTALL_DIR)
 
+unsafe-install:
+	echo "unsafe install, are you sure?"
+	read foo
+	python3 setup.py install --prefix=$(INSTALL_DIR)
+
 upload: tests upload-docs
 	python3 setup.py sdist upload
 

README.md

@@ -1,6 +1,6 @@
-`ayrton` - a shell-like scripting language based on Python3.
+`ayrton` - a shell-like scripting language strongly based on Python3.
 
-`ayrton` is an extension of the Python language that tries to make it look more
+`ayrton` is an modification of the Python language that tries to make it look more
 like a shell programming language. It takes ideas already present in `sh`, adds
 a few functions for better emulating envvars, and provides a mechanism for (semi)
 transparent remote execution via `ssh`.
@@ -54,7 +54,7 @@ So, in short:
 
 # First steps: execution, output
 
-To mimic the second example in the introduction,
+To do the same as the second example in the introduction,
 with `sh` you could `from sh import echo` and it will create a callable that will
 transparently run `/bin/echo` for you; `ayrton` takes a step further and creates
 the callable on the fly, so you don't have to pre-declare it. Another difference
@@ -73,18 +73,16 @@ Just guess were the output went :) ... (ok, ok, it went to `/dev/null`).
 
 # Composing
 
-Just like `sh`, you can nest callables, but you must explicitly tell it that you
-want to capture the output so the nesting callable gets its input:
+Just like `sh`, you can nest callables:
 
-    root= grep (cat ('/etc/passwd', _out=Capture), 'root', _out=Capture)
+    root= grep (cat ('/etc/passwd'), 'root', _out=Capture)
 
-This seems more cumbersome than `sh`, but if you think that in any shell language
-you do something similar (either using `$()`, `|` or even redirection), it's not
-a high price to pay.
+In the special case where a command is the first argument for another, its output
+will be captured and piped to the stdin of the outer command.
 
 Another improvement over `sh` is that you can use commands as conditions:
 
-    if grep (cat ('/etc/passwd', _out=Capture), 'mdione', _out=None):
+    if grep (cat ('/etc/passwd'), 'mdione', _out=None):
         print ('user «mdione» is present on your system; that's a security vulnerability right there!')
 
 As a consequence, you can also use `and`, `or` and `not`.
@@ -97,9 +95,7 @@ we had to implement it:
     if cat ('/etc/passwd') | grep ('mdione', _out=None):
         print ('I'm here, baby!')
 
-Notice that this time you don't have to be explicit about the `cat`'s output;
-we know it's going to a pipe, so we automatically `Capture` it. Of course, we
-also have redirection:
+And of course, we also have redirection:
 
     grep ('mdione') < '/etc/passwd' > '/tmp/foo'
     grep ('root') < '/etc/passwd' >> '/tmp/foo'
@@ -119,7 +115,7 @@ of:
     /home/mdione/src/projects/ayrton/bin
     /home/mdione/src/projects/ayrton
 
-`bash()` applies brace, tilde and glob (pathname) expansions:
+The `bash()` function applies brace, tilde and glob (pathname) expansions:
 
     >>> from ayrton.expansion import bash
     >>> import os
@@ -184,14 +180,17 @@ If the latter fails the construct fails and your script will finish. We're
 checking its limitations to see where we can draw the line of what will be
 possible or not.
 
+The development of this construct is not complete, so expect some changes in its
+API.
+
 Here you'll find [the docs](http://www.grulic.org.ar/~mdione/projects/ayrton/).
 
 # FAQ
 
 Q: Why bother? Isn't `bash` great?
 
-A: Yes and no. `bash` is very powerful, both from the CLI and as a language. But
-it's clumsy, mainly due to two reasons: parsing lines into commands and their
+A: Yes and no. `bash` is very powerful, both from the CLI point of view and as a language.
+But it's clumsy, mainly due to two reasons: parsing lines into commands and their
 arguments, and the methods for preventing overzealous word splitting, which leads
 to several pitfalls, some of them listed [here](http://mywiki.wooledge.org/BashPitfalls));
 and poor data manipulation syntax.  It also lacks of good remote

TODO.rst

@@ -10,6 +10,8 @@ Really do:
 
    * from foo import bar is gonna break
 
+* imported ayrton scripts should be parsed with the ayrton parser.
+
 * becareful with if cat () | grep (); error codes must be carried too
 
 * process substitution
@@ -21,7 +23,7 @@ Really do:
    * see pdb's source
 
 * becareful with buitins, might eclipse valid usages: bash() (exp) blocks /bin/bash
-
+   * rename bash() to expand()
    * add option _exec=True for actually executing the binary.
 
 * check ``bash``'s manpage and see what's missing.
@@ -31,9 +33,17 @@ Really do:
 
 * a setting for making references to unkown envvars as in bash.
 * trap?
+* executable path caching à la bash.
 
-If we {have time,are bored}:
-----------------------------
+Think deeply about:
+-------------------
 
 * what to do about relative/absolute command paths?
-* executable path caching à la bash.
+* git (commit) vs git.commit() vs git ('commit')
+* function names are expressions too:
+    * / as unary op? => /path/to/excecutable and relative/path
+    * foo_bar vs foo__bar vs foo-bar
+    * -f vs (-)f vs _f
+* commands in keywords should also be _out=Capture
+* which is the sanest default, bash (..., single=True) or otherwise
+* foo(-l, --long-option)?

ayrton/ast_pprinter.py

@@ -603,6 +603,9 @@ def pprint_inner (node, level=0):
             yield ' as '
             for i in pprint_inner (node.optional_vars): yield i
 
+    elif t==str:
+        yield node
+
     else:
         yield '\n'
         yield '# unknown construction\n'

ayrton/castt.py

@@ -50,6 +50,9 @@ def is_executable (node):
             type (node.func.func)==Name and
             node.func.func.id=='Command')
 
+def is_option (arg):
+    return type (arg)==Call and type (arg.func)==Name and arg.func.id=='o'
+
 def has_keyword (node, keyword):
     return any ([kw.arg==keyword for kw in node.keywords])
 
@@ -465,6 +468,13 @@ def visit_Call (self, node):
                         node.args.pop (0)
                         update_keyword (node, keyword (arg='_in', value=first_arg))
 
+                    for arg in node.args:
+                        if is_option (arg):
+                            # ast_pprinter takes care of expressions
+                            kw= arg.keywords[0]
+                            logger.debug ("->>>kw: %s", ast.dump (kw))
+                            kw.arg= pprint (kw.arg)
+
                 ast.copy_location (new_node, node)
                 node.func= new_node
                 ast.fix_missing_locations (node)
@@ -482,19 +492,30 @@ def visit_Call (self, node):
                 first_kw= False
                 for index, arg in enumerate (node.args):
                     # NOTE: maybe o() can be left in its own namespace so it doesn't pollute
-                    if type (arg)==Call and type (arg.func)==Name and arg.func.id=='o':
-                        kw_name= arg.keywords[0].arg
+                    if is_option (arg):
+                        kw_expr= arg.keywords[0].arg
+                        if not isinstance (kw_expr, ast.Name) and not isinstance (kw_expr, str):
+                            raise SyntaxError (self.file_name, node.lineno, node.column,
+                                               "keyword can't be an expression")
+
+                        if isinstance (kw_expr, ast.Name):
+                            kw_name= kw_expr.id
+                        else:
+                            kw_name= kw_expr  # str
+
                         if kw_name in used_keywords:
-                            raise SyntaxError(self.file_name, node.lineno, node.column,
-                                              "keyword argument repeated")
+                            raise SyntaxError (self.file_name, node.lineno, node.column,
+                                               "keyword argument repeated")
 
-                        node.keywords.append (arg.keywords[0])
+                        # convert the expr into a str
+                        new_kw= keyword (kw_name, arg.keywords[0].value)
+                        node.keywords.append (new_kw)
                         used_keywords.add (kw_name)
                         first_kw= True
                     else:
                         if first_kw:
-                            raise SyntaxError(self.file_name, node.lineno, node.column,
-                                              "non-keyword arg after keyword arg")
+                            raise SyntaxError (self.file_name, node.lineno, node.column,
+                                               "non-keyword arg after keyword arg")
 
                         new_args.append (arg)
 

ayrton/execute.py

@@ -288,16 +288,15 @@ def prepare_args (self, cmd, args, kwargs):
 
     def prepare_arg (self, seq, name, value):
         if value!=False:
-            if len (name)==1:
-                arg="-%s" % name
-            else:
-                # TODO: longopt_prefix
-                # and/or simply subclass find(Command)
-                arg="--%s" % name
-            seq.append (arg)
+            seq.append (name)
 
+            # this is not the same as 'not value'
+            # because value can have any, well, value of any kind
             if value!=True:
                 seq.append (str (value))
+        else:
+            # TODO: --no-option?
+            pass
 
     def parent (self):
         if self.stdin_pipe is not None:
@@ -355,6 +354,7 @@ def wait (self):
             self.capture_file= open (r)
 
         if self._exit_code==127:
+            # NOTE: when running bash, it returns 127 when it can't find the script to run
             raise CommandNotFound (self.path)
 
         if (ayrton.runner.options.get ('errexit', False) and

ayrton/expansion.py

@@ -41,9 +41,6 @@ def glob_expand (s):
         # accumulate them
         ans+= a
 
-    if len(ans)==1:
-        ans= ans[0]
-
     return ans
 
 class Group (object):
@@ -202,9 +199,6 @@ def brace_expand (s):
             else:
                 ans.append (te.text)
 
-    if len(ans)==1:
-        ans= ans[0]
-
     return ans
 
 def backslash_descape (s):
@@ -225,5 +219,8 @@ def tilde_expand (s):
 
     return ans
 
-def bash (s):
-    return backslash_descape (glob_expand (tilde_expand (brace_expand (s))))
+def bash (s, single=False):
+    data= backslash_descape (glob_expand (tilde_expand (brace_expand (s))))
+    if single and len(data)==1:
+        data= data[0]
+    return data

ayrton/parser/astcompiler/astbuilder.py

@@ -1187,9 +1187,6 @@ def handle_call(self, args_node, callable_expr):
             if argument.type == syms.argument:
                 if len(argument.children) == 1:
                     expr_node = argument.children[0]
-                    # if keywords:
-                    #     self.error("non-keyword arg after keyword arg",
-                    #                expr_node)
                     if variable_arg:
                         self.error("only named arguments may follow "
                                    "*expression", expr_node)
@@ -1202,21 +1199,22 @@ def handle_call(self, args_node, callable_expr):
                     if isinstance(keyword_expr, ast.Lambda):
                         self.error("lambda cannot contain assignment",
                                    keyword_node)
-                    elif not isinstance(keyword_expr, ast.Name):
-                        self.error("keyword can't be an expression",
-                                   keyword_node)
-                    keyword = keyword_expr.id
-                    self.check_forbidden_name(keyword, keyword_node)
+                    keyword = keyword_expr
+                    if isinstance (keyword, ast.Name):
+                        self.check_forbidden_name(keyword.id, keyword_node)
                     keyword_value = self.handle_expr(argument.children[2])
-                    if keyword in Command.supported_options:
-                        keywords.append(ast.keyword(keyword, keyword_value))
+                    if isinstance (keyword, ast.Name) and keyword.id in Command.supported_options:
+                        keywords.append(ast.keyword(keyword.id, keyword_value))
                     else:
                         kw = ast.keyword(keyword, keyword_value)
                         kw.lineno = keyword_node.lineno
+                        kw.col_offset = keyword_node.column
                         name = ast.Name ('o', ast.Load())
                         name.lineno = keyword_node.lineno
+                        name.column = keyword_node.column
                         arg = ast.Call(name, [], [ kw ], None, None)
                         arg.lineno = keyword_node.lineno
+                        arg.column = keyword.column
                         args.append(arg)
             elif argument.type == tokens.STAR:
                 variable_arg = self.handle_expr(args_node.children[i + 1])

ayrton/tests/test_ayrton.py

@@ -35,10 +35,16 @@
 
 class Bash(unittest.TestCase):
     def test_simple_string (self):
-        self.assertEqual (bash ('s'), 's')
+        self.assertEqual (bash ('s'), [ 's' ])
+
+    def test_simple_string_single (self):
+        self.assertEqual (bash ('s', single=True), 's')
 
     def test_glob1 (self):
-        self.assertEqual (bash ('*.py'), 'setup.py')
+        self.assertEqual (bash ('*.py'), [ 'setup.py' ])
+
+    def test_glob1_single (self):
+        self.assertEqual (bash ('*.py', single=True), 'setup.py')
 
     def test_glob2 (self):
         self.assertEqual (sorted (bash ([ '*.py', '*.txt' ])), [ 'LICENSE.txt', 'setup.py', ])
@@ -56,10 +62,16 @@ def test_simple2_brace (self):
         self.assertEqual (bash ('a{b,ce}d'), [ 'abd', 'aced' ])
 
     def test_simple3_brace (self):
-        self.assertEqual (bash ('{a}'), '{a}')
+        self.assertEqual (bash ('{a}'), [ '{a}' ])
+
+    def test_simple3_brace_single (self):
+        self.assertEqual (bash ('{a}', single=True), '{a}')
 
     def test_simple4_brace (self):
-        self.assertEqual (bash ('a}'), 'a}')
+        self.assertEqual (bash ('a}'), [ 'a}' ])
+
+    def test_simple4_brace_single (self):
+        self.assertEqual (bash ('a}', single=True), 'a}')
 
     def test_simple5_brace (self):
         self.assertEqual (bash ('a{bfgh,{ci,djkl}e'), [ 'a{bfgh,cie', 'a{bfgh,djkle' ])
@@ -78,14 +90,20 @@ def test_nested2_brace (self):
         self.assertEqual (bash ('{c{a,b}d,e{f,g}h}'), [ 'cad', 'cbd', 'efh', 'egh' ])
 
     def test_escaped_brace (self):
-        self.assertEqual (bash ('\{a,b}'), '{a,b}')
+        self.assertEqual (bash ('\{a,b}'), [ '{a,b}' ])
+
+    def test_escaped_brace_single (self):
+        self.assertEqual (bash ('\{a,b}', single=True), '{a,b}')
 
     def test_real_example1 (self):
         # tiles/{legend*,Elevation.dgml,preview.png,Makefile}
         pass
 
     def test_tilde (self):
-        self.assertEqual (bash ('~'), os.environ['HOME'])
+        self.assertEqual (bash ('~'), [ os.environ['HOME'] ])
+
+    def test_tilde_single (self):
+        self.assertEqual (bash ('~', single=True), os.environ['HOME'])
 
 def setUpMockStdout (self):
     # due to the interaction between file descriptors,
@@ -203,7 +221,7 @@ def testPipe (self):
         self.assertEqual (self.r.read (), b'setup.py\n')
 
     def testLongPipe (self):
-        ayrton.main ('ls () | grep ("setup") | wc (l=True)')
+        ayrton.main ('ls () | grep ("setup") | wc (-l=True)')
         # close stdout as per the description of setUpMockStdout()
         os.close (1)
         self.assertEqual (self.r.read (), b'1\n')