Python Shell C C++ JavaScript HTML
Permalink
Failed to load latest commit information.
asdl More language elements handled in osh-to-oil. Jan 27, 2017
bin More language elements handled in osh-to-oil. Jan 27, 2017
core Make shebang lines consistent. (This matters on OS X.) Feb 14, 2017
oil Copy from sketch repo. This passes: Nov 16, 2016
osh Make shebang lines consistent. (This matters on OS X.) Feb 14, 2017
ovm More language elements handled in osh-to-oil. Jan 27, 2017
tests More work converting osh to oil. Feb 3, 2017
web Progress on osh to oil conversion. Feb 8, 2017
.gitignore Ignore Python build output Nov 19, 2016
LICENSE Initial commit Nov 15, 2016
README.md Tweak the 'hello world' example. Dec 7, 2016
__init__.py Start integrating ASDL into the codebase. Dec 17, 2016
count.sh Get osh_demo compiling with id_kind.h. Hard-coding the Id type for now. Jan 7, 2017
lint.sh Various lint fixes with pep8. Nov 22, 2016
osh-to-oil-test.sh Progress on osh to oil conversion. Feb 8, 2017
pybuild.sh Rename function Nov 19, 2016
refactor.sh Fixing lint errors Nov 22, 2016
release.sh Start printing osh to oil. Jan 20, 2017
run.sh Add a file comment for all top-level shell scripts Nov 19, 2016
setup.py Build Python extension. All unit tests pass. Nov 18, 2016
sh_spec.py Convert sh_spec.py to Python 3 Nov 19, 2016
sh_spec_test.py Convert sh_spec.py to Python 3 Nov 19, 2016
smoke.sh Start printing osh to oil. Jan 20, 2017
spec.sh Start printing osh to oil. Jan 20, 2017
test.sh Fix ASDL type checking. Jan 4, 2017
wild.sh Progress on osh to oil conversion. Feb 8, 2017

README.md

oil

Oil is a new Unix shell, still in its early stages.

This repo contains a prototype in Python of a very complete bash parser, along with a runtime that is less complete.

The dialect of bash that is recognized is called the osh language. The main goal now is to design the oil language, which shell scripts can be automatically converted to.

After that, the Python dependency can be broken by porting it to C++.

Try it

Clone the repo and run bin/osh. Basic things like pipelines, variables, functions, etc. should work.

bash$ bin/osh
osh$ name=world
osh$ echo "hello $name"
hello world

Build it

Python's builtin glob and fnmatch modules don't match libc in some cases (e.g. character classes). To fix that, build the core/libc.c wrapper:

$ ./pybuild.sh libc

Now bin/osh will use libc's globbing.

Running Tests

There are three kinds of tests: unit tests, spec tests, and "wild tests".

Unit tests are in Python:

$ ./test.sh all-unit
$ ./test.sh unit osh/word_parse_test.py

(test.sh is a simple wrapper that sets PYTHONPATH)

Spec tests are written with the sh_spec.py framework:

$ ./spec.sh setup
$ ./spec.sh smoke   # or other actions

"Wild tests" test the parser against code in the wild. We don't have any golden data to compare against, but whether the parse succeeds or fails is useful for shaking out bugs, sort of like a fuzz test.

$ ./wild.sh this-repo

This will run the parser on shell scripts in this repo, and put the output in _tmp/wild/oil-parsed, which you can view with a web browser.

Code Overview

Try this to show a summary of what's in the repo and their line counts:

$ ./count.sh all

(Other functions in this file that may be useful as well.)

Directory Structure

bin/              # programs to run (bin/osh)
core/             # the implementation (AST, runtime, etc.)
osh/              # osh front end
oil/              # oil front end (empty now)
tests/            # spec tests

pybuild.sh        # build support
setup.py

test.sh           # test scripts
spec.sh
wild.sh
smoke.sh
sh_spec.py        # shell test framework

lint.sh           # static analysis
typecheck.sh

count.sh          # Get an overview of the repo

_tmp/             # For test temp files

Unit tests are named foo_test.py and live next to foo.py.

More info

If you need help using oil, or have general questions, e-mail oil-discuss@oilshell.org.

If you want to contribute, e-mail oil-dev@oilshell.org.

I have docs that need to be cleaned up and published. For now, there is a fair amount of design information on the blog at oilshell.org.