oilshell/oil

Instead of 'opyc dis' taking an optional report_path, we now have 'opyc
dis-tables', which outputs 5 tables related to Python's
types.CodeObject:

- frames
- names
- consts
- ops
- flags

Some ad hoc analysis in R already produced some interesting results,
e.g. how big the instructions are (~216 KB) vs. all the .pyc files
(~918K).  The rests is constants, metadata, .pyc headers, etc.

Also: Make an intermediate file _build/oil/all-deps-py.txt.  Right now
we're counting the lines with it, but it make it easier to do metrics in
general.  (e.g. compilation time)

Also make 0.6.pre4 the latest download on the home page.

Unrelated:

- Remove unnecessary code in osh2oil.
- Minor cleanups of imports

When 'a' in (( a++ )) is an array.

Still needs location info.

OSH is intentionally stricter in two ways:

- arrays can't be used as environment bindings!
- strings can't be indexed by integers; only arrays

It seems like has to have another redirect after it.  Example:

git/t/t4015-diff-whitespace.sh

The translation no longer crashes, but it's still wrong.

The Eof_Real token was being treated as a substitute for Eof_RParen,
which was invalid.

Fixes issue #144.

- Notes about the span representation.
- Update some comments.
- 'oshc spans' prints to stdout, no stderr.

- for something like 'echo a(b)'.
- for something like 'do echo hi'

This fixed spec tests in more than one file.

- Avoid a crash in the special case of the empty file.
  - The line_id is -1 only in that case, which makes logical sense, but
    caused a crash.
  - We might want to have no zero-length spans then?  Or say that
    Eof_Real doesn't produce a span?

- Don't crash on ExtGlobPart.  Though I'm not sure exactly how to
  translate this.

Also: categorize a few more wild errors.

Uncovered by wild tests.

This was caused by the main_loop refactoring.

Example of file that was crashing, and now parses:

illumos-gate/usr/src/lib/libshell/common/tests/expand.sh

Added test cases.

Also:

- Combine parsing and tree representation of while and until loops.  The
  execution was already unified.
- Fix the translation of while and until conditions, and add a test.
  while (( x == 0 )); now properly turns into while sh-expr ' x == 0 '.

It will compile it first and then disassemble it.

The gold test somehow doesn't trigger the errexit behavior?

gold/errexit-confusion.sh: A confusing case in bash execution.  The
arena test is not failing properly.

- Minor update to release index
- Update OPy regtest checksums.

Found some errors: parser bugs tickled by 'wild' harness, arena test,
opy regtest, etc.

- builtins: alias, unalias, compgen (partial)
- modes: strict-word-eval (done), strict-var-eval (speculative)
- latest ideas about Oil
  - including all the compatibility shims like sh-expr

This should be the last of it.

Get rid of Error() and error_stack.  We're using exceptions now.

- Remove old comments
- Rename back to VirtualLineReader

Down to 6 failures now, which are all related to aliasing grammatical
constructs other than SimpleComamand.

Unrealted: removed unused code in bool_parse.py

This is mainly motivated by 'alias'.  Aliases are unfortunately truly
global state.

It also breaks some longstanding circular imports between word_parse.py
and cmd_parse.py, which is nice.

All unit, spec, and gold tests pass.  Still need to do some cleanup.

Remove some other unused code, and add a comment about another approach
that is more like dash and bash.

The previous approach of PushAliasBuffer() only worked for a single
alias expansion.  Iteration was a problem.

Now we hook into ParseSimpleCommand and expand aliases iteratively.
We join the expanded aliases with the rest of the *text* from the
SimpleCommand, and then pass that into ParseCommand().

I made some comments about the cases that this doesn't handle.  And also
some notes about how dash and bash do it -- with global variables strewn
throughout the parser.

(Although I wonder if we could do that with self._Next() ? )

- Add more alias test cases
- Factor some free functions out of CommandParser.

All unit, spec, and gold tests pass.

This isn't totally correct since multiple alias expansion aren't
handled.  But we go from 22 test cases failing to 6!

The alias expansion is detected in CommandParser.ParseCommand(), and
then we push a buffer onto the Lexer at that point.  This seems to be
the correct place to do it.

Addresses issue #160.

Use args.UsageError more consistently through core/builtin.py.

All spec tests pass.

Add more spec test cases for alias.

- Reorder methods.
- Rename ParseOne -> ParseLogicalLine.
- Make some methods private (more could be done.)
  - ParseLogicalLine() and ParseCommandSub() are really the only public
    methods.  The main_loop.ParseWholeFile() is a wrapper around this
    public interface.

- word_parse.py uses c_parser.ParseCommandSub() for $() and ``
- source and eval use main_loop.Batch() to interleave parsing and
  execution (e.g. so they can have aliases defined.)
- ParseTrap uses main_loop.ParseWholeFile()
- oshc also uses main_loop.ParseWholeFile()

All unit, spec, gold, and osh2oil tests pass.

- Add util.Debug() which logs to a file specified by --debug-file.
- Add configure-bug to the list of passing.

All gold, spec, and unit tests pass!  Finally.