release-quality.md

css_files
web/base.css web/toc.css web/release-index.css

Oil 0.9.7 Quality

This is a supplement to the release index.

Test Results

Spec Tests

• OSH Survey. Test OSH with existing shells, and compare their behavior.
• Stateful Tests. Tests that use pexpect.
• OSH in C++. The progress of Oil's C++ translation.
• Oil Language. The legacy-free language.

Other Primary Suites

• Gold Tests. Compare OSH against bash (using implicit assertions, no golden output.)
• Wild Tests. Parse and translate thousands of shell scripts with OSH.
• Unit Tests. Python unit test results.

More

• Smoosh test suite (from mgree/smoosh):
  • smoosh | smoosh-hang
• parse-errors. A list of all parse errors.
  • parse-errors-oil-native. With the native binary.
• runtime-errors. A list of shell runtime errors.
• oil-runtime-errors. Oil language runtime errors.
• oil-large
• tea-large
• arena. Testing an invariant for the parser.
• osh-usage. Misc tests of the osh binary.
• oshc-deps. Tests for a subcommand in progress.
• How many processes does Oil start compared to other shells?
  • syscall/by-code | syscall/by-input
• osh2oil Tests. Test the conversion of OSH to Oil.

Benchmarks

• Parser. How fast does OSH parse compared to other shells?
• Runtime. How fast does OSH run shell scripts?
• Compute. How fast does OSH run small programs without I/O?
• Build. How long does it take for end users to build Oil? How big is the resulting binary?
• Virtual Memory Baseline. How much memory do shells use at startup?
• mycpp. Compares Python and generated C++ on small examples.

Metrics

• Lines of source, counted in different ways:
  • overview. The whole Oil repo organized by type of source file.
  • for-translation. An overview of the "compile engineer" project.
  • osh-cloc. OSH and common libraries, as measured by the cloc tool.
  • oil-cpp. The C++ code in the oil-native tarball.
• oil-native.
  • overview - Binary code size reported by Bloaty.

Source Code

These files may help you understand how Oil is implemented, i.e. with domain-specific languages and code generation.

• _devbuild/gen/id.h. A list of language elements, used in the lexer and in multiple parsers and evaluators.
• The regex-based lexer uses two stages of code generation:
  • frontend/lexer_def.py | _devbuild/tmp/osh-lex.re2c.h | _devbuild/gen/osh-lex.h
• frontend/syntax.asdl. The syntax tree for OSH and Oil.
• oil_lang/grammar.pgen2. The expression grammar for Oil. In contrast, the OSH parsers are hand-written.

Also see the oilshell/oil repository.

Old

These links describe the CPython / "OVM" build, which should become the "experimental" version of Oil.

OPy Tests

The OPy compiler is used to compile Oil to bytecode, but isn't itself part of the release.

• build-oil-repo | test-gold | regtest-compile | regtest-verify-golden
• Tree-shaking: Symbols in Oil | Symbols in OPy
• other/opyc

OPy / OVM Metrics

• OPy Line Counts.
• Lines of dependencies:
  • pydeps. Oil code plus the Python standard library.
  • nativedeps. Oil code plus a slice of CPython.
• Bytecode Metrics
  • overview - Compare OPy vs. CPython.
  • oil-with-opy - Oil compiled with OPy.
  • oil-with-cpython - Oil compiled with CPython (for comparison).
  • src-bin-ratio-with-opy - How big is the compiled output?
• OVM / CPython
  • overview - An analysis of GCC's compilation of OVM (a subset of CPython). Bloaty provides the underlying data.
  • cpython-defs/overview - We try to ship as little of CPython as possible, and this is what's left.