Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Need better regression tests for the optimizer #124

Open
wingo opened this issue Feb 3, 2015 · 4 comments
Open

Need better regression tests for the optimizer #124

wingo opened this issue Feb 3, 2015 · 4 comments

Comments

@wingo
Copy link
Contributor

wingo commented Feb 3, 2015

Sometimes we find optimizer bugs that have a precise test case we can run. It would be good to be able to drop input and expected output files into a directory so we can test; e.g.

regression-tests/foo

#!/bin/sh
pflua-optimize <<END
{ "if",
  { "!=",
    "len",
    0 },
  { ">",
    1,
    { "uint32",
      { "+",
        { "uint32",
          { "-",
            4294967295,
            { "uint32",
              { "/",
                1,
                "len" } } } },
        1 } } },
  { "fail" } }
END

regression-tests/foo.expected

Optimizing:
{ "if",
  { "!=",
    "len",
    0 },
  { ">",
    1,
    { "uint32",
      { "+",
        { "uint32",
          { "-",
            4294967295,
            { "uint32",
              { "/",
                1,
                "len" } } } },
        1 } } },
  { "fail" } }
Result:
{ "if",
  { "!=",
    "len",
    0 },
  { "<",
    { "uint32",
      { "+",
        { "-",
          4294967295,
          { "/",
            1,
            "len" } },
        1 } },
    1 },
  { "fail" } }

Then the harness would run the input file, as an executable, and cmp its output to the expected output. If the optimizer changes we might get spurious diffs but we should be seeing the changes, and a simple makefile rule can regenerate expected output. Dunno. Point being, we should let the property tester find the bugs, and the regression tests keep them from happening again.

@kbara
Copy link
Contributor

kbara commented Feb 3, 2015

Yes; we need unit tests for a handful of known bugs that we've fixed, whether or not we fit them into the regressions test framework.

@wingo
Copy link
Contributor Author

wingo commented Feb 3, 2015

@kbara would you mind hacking up something for this? that would be swell :)

@kbara
Copy link
Contributor

kbara commented Feb 3, 2015

I'll think it over, then hack something up, yes.

@kbara
Copy link
Contributor

kbara commented May 13, 2015

@andywingo Are you happy with the state of tests/ir-reg/* for this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants