Contributing

NOTE: If anything on this page doesn't work for you, please file a bug!

Requirements

I want Oil to work on any Unix including OS X, but right now development is best done on Linux. This is mainly because the tests run against four other shells (bash/dash/zsh/mksh), and I personally haven't ever run those shells on OS X.

Quick Start

git clone https://github.com/oilshell/oil.git    # or use your own fork

cd oil                    # or enter the base directory

build/dev.sh ubuntu-deps  # Python 2 dev headers, etc.
                          # Uses sudo apt-get, so it's for Ubuntu/Debian.
build/dev.sh minimal      # build help, native/libc.c Python extension

bin/osh                   # sanity check to make sure it works. 
                          # Make sure you have Python 2 installed.
                          # Ctrl-D to quit.

mkdir -p _tmp             # This directory tree is used by tests.

test/unit.sh unit core/lexer_test.py     # Run an individual unit test
test/unit.sh all          # run all unit tests, about a second or two

test/spec.sh smoke        # run a single file -- see spec tests in parallel
test/spec.sh all          # run all spec tests in parallel, ~11 seconds on 
                          # a fast i7 desktop. Should be done in about 30s.

After you've run these tests, you're ready to work on Oil! Details below.

Running Three Kinds of Tests

There are three kinds of tests: unit tests, spec tests, and "wild tests".

Unit tests are in Python:

$ ./unit.sh all
$ ./unit.sh one osh/word_parse_test.py

(unit.sh is a simple wrapper that sets PYTHONPATH)

Spec Tests are written with the sh_spec.py framework:

$ ./spec.sh install-shells
$ ./spec.sh all     # all in parallel
$ ./spec.sh smoke   # a single file -- look at the list of functions

Output is in _tmp/spec.

"Wild tests" test the parser against code in the wild. We don't have any golden data to compare against, but whether the parse succeeds or fails is useful for shaking out bugs, sort of like a fuzz test.

$ ./wild.sh this-repo

This will run the parser on shell scripts in this repo, and put the output in _tmp/wild/oil-parsed, which you can view with a web browser.

Three Kinds of Builds

(1) What I've documented above, i.e. running bin/osh, can be called the developer build. This builds native/libc.c and uses the system Python interpreter /usr/bin/env python to run Oil. It may go away as we transition to OPy, because Oil may no longer run on a raw Python interpreter.

(2) The "production" build is split into two steps: (a) git repo -> release tarball, and (b) release tarball -> Oil app bundle.

2a. Release tarball. This step is architecture-independent and occurs on the developer machine. This process may change, but it's roughly as follows:

# First we need to build a raw Python interpreter, in order to dynamically discover Python dependencies.
build/dev.sh ubuntu-deps        # or equivalent for other distros
build/prepare.sh configure      # configure CPython
build/prepare.sh build-python   # compile CPython
build/codegen.sh download-re2c  
build/codegen.sh install-re2c
build/dev.sh all
scripts/release.sh oil          # build _release/oil.tar with the Makefile and build/*.{sh,py}
build/test.sh oil-tar           # test that you can build the tarball

NOTE: This step is likely to be Linux-centric for awhile. The end-user build, step 2b, should not be Linux-centric. Ideally it would only require a POSIX environment.

2b. Oil app bundle build. This step is architecture-dependent and occurs on the machine of the system packager or end user. To perform this step, follow the instructions in the INSTALL.txt file at the root of the tarball.

OPy Compiler and Byterun

See opy/README.md.

What Are Spec Tests?

See Spec Tests