Skip to content

Commit

Permalink
Test helper: Mark current run as reference
Browse files Browse the repository at this point in the history
When developing on edalize, it's very common to make changes which
affect the generated files. To update the tests, one then needs to
manually apply the changes to the generated files, or step-by-step copy
over the generated tests from the failing comparison to the reference
location.

This commit adds a way to automate this process by marking a pytest run
as "reference run", and using the generated files as reference file.

To mark a run as reference run, set the GOLDEN_RUN environment variable
to any value, e.g.

```
GOLDEN_RUN=1 pytest
```

Don't forget to use git diff to then compare if the changes are actually
expected!

Implementation note:
This uses an environment variable and not a command line argument for
ease of implementation. Command line arguments needs to be passed as
fixture in pytest, which requires changes to every test_* function we
have, or ugly workarounds with globals.
  • Loading branch information
imphil authored and olofk committed Jul 2, 2019
1 parent 0f0b4fb commit 2030196
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions tests/edalize_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,21 @@

def compare_files(ref_dir, work_root, files):
import os.path
import shutil

for f in files:
reference_file = os.path.join(ref_dir, f)
generated_file = os.path.join(work_root, f)

assert os.path.exists(generated_file)

if 'GOLDEN_RUN' in os.environ:
shutil.copy(generated_file, reference_file)

with open(reference_file) as fref, open(generated_file) as fgen:
assert fref.read() == fgen.read(), f


def param_gen(paramtypes):
args = []
defs = {}
Expand Down

0 comments on commit 2030196

Please sign in to comment.