Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 58 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,60 @@
# FileCheck.py

Attempt to reimplement LLVM's FileCheck using Python.

## Background

We know at least two software projects that would benefit from a suite of
LLVM LIT integration tests:

1. [Mull project](https://github.com/mull-project/mull)

2. [Doorstop](https://github.com/doorstop-dev/doorstop/pull/431)

The problem is that you need to have to build `FileCheck` from LLVM sources
which is not a trivial task for 1) people who are not aware with LLVM
infrastructure and 2) Python-based projects that would prefer to not have
to build anything from LLVM sources in their CI process.

The option of having pre-compiled binaries is a workaround, but we don't like to
keep third-party binary artifacts in source code,
(see https://github.com/doorstop-dev/doorstop/pull/431#issuecomment-549237579).

## Goals

The first goal is to make FileCheck.py pass on Mull's current integration
tests suite which uses only a very limited subset of FileCheck's features which
are as follows:

- Commands (both substring and regex matching):
- `CHECK`
- `CHECK-NEXT`
- `CHECK-NOT`
- `CHECK-EMPTY`
- Options:
- `--strict-whitespace`
- `--match-full-lines`
- `--check-prefix`

When this is done, it feels like a good idea to implement full FileCheck's
contract and be 100% compatible with the C++ version.

There is a file in this repository: [FileCheck.pdf](FileCheck.pdf) which has
some rough implementation coverage: green color means something is implemented,
yellow color means that the text is not relevant to implementation, no
highlighting means "still to be implemented".

## Notes on implementation

The implementation is made against the latest
[FileCheck - Flexible pattern matching file verifier](https://llvm.org/docs/CommandGuide/FileCheck.html)
document. The plan is to skip looking into the LLVM's implementation of
FileCheck, unless some very advanced implementation details are encountered.

It a nice programming exercise of implementing something from a spec but we also
want to make FileCheck.py to conform to the same contract this is why
FileCheck.py is tested using LIT and the LLVM's FileCheck:

First, real FileCheck is run against a LIT/FileCheck test suite, then
FileCheck.py is run on the same tests to make sure that it has the same behavior
as its LLVM FileCheck parent.