Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[release] Create release/$VERSION/quality.html page
- Loading branch information
Andy C
committed
Feb 19, 2022
1 parent
92462f4
commit 926e6b5
Showing
4 changed files
with
184 additions
and
159 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,167 @@ | ||
| --- | ||
| css_files: web/base.css web/toc.css web/release-index.css | ||
| --- | ||
|
|
||
| Oil 0.9.7 Quality | ||
| ================= | ||
|
|
||
| <!-- NOTE: This file is published to /release/$VERSION/quality.html --> | ||
|
|
||
| <span class="date"> | ||
| <!-- REPLACE_WITH_DATE --> | ||
| </span> | ||
|
|
||
| This is a supplement to the [release index](index.html). | ||
|
|
||
| <div id="toc"> | ||
| </div> | ||
|
|
||
| ## Test Results | ||
|
|
||
| ### Spec Tests | ||
|
|
||
| - [OSH Survey](test/spec.wwz/survey/osh.html). Test OSH with existing shells, | ||
| and compare their behavior. | ||
| - [Stateful Tests](test/spec.wwz/stateful/index.html). Tests that use | ||
| [pexpect]($xref). | ||
| - [OSH in C++](test/spec.wwz/cpp/osh-summary.html). The progress of Oil's C++ | ||
| translation. | ||
| - [Oil Language](test/spec.wwz/oil-language/oil.html). The legacy-free | ||
| language. | ||
|
|
||
| ### Other Primary Suites | ||
|
|
||
| - [Gold Tests](test/other.wwz/gold.txt). Compare OSH against bash (using | ||
| implicit assertions, no golden output.) | ||
| - [Wild Tests](test/wild.wwz/). Parse and translate thousands of shell scripts | ||
| with OSH. | ||
| - [Unit Tests](test/unit.wwz/). Python unit test results. | ||
|
|
||
| ### More | ||
|
|
||
| - [Smoosh][] test suite (from [mgree/smoosh][]): | ||
| - [smoosh](test/spec.wwz/survey/smoosh.html) | ||
| | [smoosh-hang](test/spec.wwz/survey/smoosh-hang.html) | ||
| - [parse-errors](test/other.wwz/parse-errors.txt). A list of all parse errors. | ||
| - [parse-errors-oil-native](test/other.wwz/parse-errors-oil-native.txt). | ||
| With the native binary. | ||
| - [runtime-errors](test/other.wwz/runtime-errors.txt). A list of shell runtime | ||
| errors. | ||
| - [oil-runtime-errors](test/other.wwz/oil-runtime-errors.txt). Oil language | ||
| runtime errors. | ||
| - [oil-large](test/other.wwz/oil-large.txt) | ||
| - [tea-large](test/other.wwz/tea-large.txt) | ||
| - [arena](test/other.wwz/arena.txt). Testing an invariant for the parser. | ||
| - [osh-usage](test/other.wwz/osh-usage.txt). Misc tests of the `osh` binary. | ||
| - [oshc-deps](test/other.wwz/oshc-deps.txt). Tests for a subcommand in | ||
| progress. | ||
| - How many processes does Oil start compared to other shells? | ||
| - [syscall/by-code](test/other.wwz/syscall/by-code.txt) | ||
| | [syscall/by-input](test/other.wwz/syscall/by-input.txt) | ||
| - [osh2oil Tests](test/other.wwz/osh2oil.txt). Test the conversion of OSH to | ||
| Oil. | ||
|
|
||
| [Smoosh]: http://shell.cs.pomona.edu/ | ||
|
|
||
| [mgree/smoosh]: https://github.com/mgree/smoosh/tree/master/tests/shell | ||
|
|
||
| ## Benchmarks | ||
|
|
||
| - [Parser](benchmarks.wwz/osh-parser/). How fast does OSH | ||
| parse compared to other shells? | ||
| - [Runtime](benchmarks.wwz/osh-runtime/). How fast does OSH run shell | ||
| scripts? | ||
| - [Compute](benchmarks.wwz/compute/). How fast does OSH run small programs | ||
| without I/O? | ||
| - [Build](benchmarks.wwz/ovm-build/). How long does it take for end users to | ||
| build Oil? How big is the resulting binary? | ||
| - [Virtual Memory Baseline](benchmarks.wwz/vm-baseline/). How much memory do | ||
| shells use at startup? | ||
| - [mycpp](benchmarks.wwz/mycpp-examples/). Compares Python and generated C++ | ||
| on small examples. | ||
|
|
||
| ## Metrics | ||
|
|
||
| - Lines of source, counted in different ways: | ||
| - [overview](pub/metrics.wwz/line-counts/overview.html). The whole Oil repo organized by | ||
| type of source file. | ||
| - [for-translation](pub/metrics.wwz/line-counts/for-translation.html). | ||
| An overview of the "compile engineer" project. | ||
| - [osh-cloc](pub/metrics.wwz/line-counts/osh-cloc.txt). OSH and common | ||
| libraries, as measured by the [cloc][] tool. | ||
| - [oil-cpp](pub/metrics.wwz/line-counts/oil-cpp.txt). The C++ code in the | ||
| `oil-native` tarball. | ||
| - `oil-native`. | ||
| - [overview](pub/metrics.wwz/oil-native/overview.txt) - Binary code size reported | ||
| by [Bloaty][]. | ||
|
|
||
|
|
||
| [cloc]: https://github.com/AlDanial/cloc | ||
| [Bloaty]: https://github.com/google/bloaty | ||
| [OVM]: //www.oilshell.org/cross-ref.html?tag=OVM#OVM | ||
|
|
||
| ## Source Code | ||
|
|
||
| These files may help you understand how Oil is implemented, i.e. with | ||
| domain-specific languages and code generation. | ||
|
|
||
| - [_devbuild/gen/id.h](source-code.wwz/_devbuild/gen/id.h). A list of language | ||
| elements, used in the lexer and in multiple parsers and evaluators. | ||
| - The regex-based lexer uses two stages of code generation: | ||
| - [frontend/lexer_def.py](source-code.wwz/frontend/lexer_def.py) | ||
| | [_devbuild/tmp/osh-lex.re2c.h](source-code.wwz/_devbuild/tmp/osh-lex.re2c.h) | ||
| | [_devbuild/gen/osh-lex.h](source-code.wwz/_devbuild/gen/osh-lex.h) | ||
| - [frontend/syntax.asdl](source-code.wwz/frontend/syntax.asdl). The syntax tree | ||
| for OSH and Oil. | ||
| - [oil_lang/grammar.pgen2](source-code.wwz/oil_lang/grammar.pgen2). The | ||
| expression grammar for Oil. In contrast, the OSH parsers are hand-written. | ||
|
|
||
| Also see the [oilshell/oil](https://github.com/oilshell/oil) repository. | ||
|
|
||
| <!-- - [OHeap](benchmarks.wwz/oheap/). Metrics for a possible AST encoding format. --> | ||
|
|
||
| <!-- TODO: | ||
| /src/ annotated/cross-referenced source code | ||
| coverage/ code coverage in Python and C | ||
| --> | ||
|
|
||
| ## Old | ||
|
|
||
| These links describe the CPython / "[OVM]($xref)" build, which should become | ||
| the "experimental" version of Oil. | ||
|
|
||
| #### OPy Tests | ||
|
|
||
| The OPy compiler is used to compile Oil to bytecode, but isn't itself part of | ||
| the release. | ||
|
|
||
| - [build-oil-repo](test/opy.wwz/build-oil-repo.txt) | ||
| | [test-gold](test/opy.wwz/test-gold.txt) | ||
| | [regtest-compile](test/opy.wwz/regtest-compile.txt) | ||
| | [regtest-verify-golden](test/opy.wwz/regtest-verify-golden.txt) | ||
| - Tree-shaking: [Symbols in Oil](test/opy.wwz/oil-symbols.txt) | ||
| | [Symbols in OPy](test/opy.wwz/opy-symbols.txt) | ||
| - [other/opyc](test/other.wwz/opyc.txt) | ||
|
|
||
| #### OPy / OVM Metrics | ||
|
|
||
| - [OPy Line Counts](pub/metrics.wwz/line-counts/opy.txt). | ||
| - Lines of dependencies: | ||
| - [pydeps](pub/metrics.wwz/line-counts/pydeps.txt). Oil code plus the Python | ||
| standard library. | ||
| - [nativedeps](pub/metrics.wwz/line-counts/nativedeps.txt). Oil code plus a | ||
| slice of CPython. | ||
| - Bytecode Metrics | ||
| - [overview](pub/metrics.wwz/bytecode/overview.txt) - Compare OPy vs. CPython. | ||
| - [oil-with-opy](pub/metrics.wwz/bytecode/oil-with-opy.txt) - Oil compiled with | ||
| OPy. | ||
| - [oil-with-cpython](pub/metrics.wwz/bytecode/oil-with-cpython.txt) - Oil | ||
| compiled with CPython (for comparison). | ||
| - [src-bin-ratio-with-opy](pub/metrics.wwz/bytecode/src-bin-ratio-with-opy.txt) - | ||
| How big is the compiled output? | ||
| - OVM / CPython | ||
| - [overview](pub/metrics.wwz/ovm/overview.txt) - An analysis of GCC's | ||
| compilation of [OVM][] (a subset of CPython). [Bloaty][] provides the | ||
| underlying data. | ||
| - [cpython-defs/overview](pub/metrics.wwz/cpython-defs/overview.txt) - We try to | ||
| ship as little of CPython as possible, and this is what's left. |