Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testsuite: extend the make one target #1530

Closed
wants to merge 1 commit into
base: trunk
from

Conversation

Projects
None yet
2 participants
@shindere
Copy link
Contributor

shindere commented Dec 14, 2017

Until now, the make one DIR=... command was able to handle only
directories containing legacy tests.

This PR extends the target so that it can also handle directories
containing ocamltest-driven tests.

testsuite: extend the make one target
Until now, the make one DIR=... command was able to handle only
directories containing legacy tests.

This commit extends the target so that it can also handle directories
containing ocamltest-driven tests.
@gasche

This comment has been minimized.

Copy link
Member

gasche commented Dec 14, 2017

Thanks a lot, I was thinking of making this PR today but I didn't find the time.

@gasche

This comment has been minimized.

Copy link
Member

gasche commented Dec 14, 2017

Thanks, I was thinking of making this PR today but I didn't find the time.

This is good to merge if the CI approves... but it looks like the CI is not happy.

@gasche

This comment has been minimized.

Copy link
Member

gasche commented Dec 14, 2017

Summary of the CI output:

[...]
 ... testing 'w58.opt.ml' with ocamlopt: => passed
 ... testing 'w55.opt_backend.ml' with ocamlopt: => passed
 ... testing 'w59.opt_backend.ml' with ocamlopt: => passed
Running tests from 'tests/win-unicode' ...
 ... testing 'exec_tests.ml': => skipped
 ... testing 'mltest.ml': => skipped
 ... testing 'symlink_tests.ml': => skipped
make[2]: *** [retries] Error 3
make[1]: *** [legacy-without-report] Error 2
make: *** [all] Error 2

(all tests reported in the log are skipped or a success.)

@shindere

This comment has been minimized.

Copy link
Contributor Author

shindere commented Dec 14, 2017

@shindere

This comment has been minimized.

Copy link
Contributor Author

shindere commented Dec 14, 2017

@shindere

This comment has been minimized.

Copy link
Contributor Author

shindere commented Dec 14, 2017

Some help would be welcome on that one:

  1. Is there a way to run AppVeyor again, to make sure the build
    didn't fail randomly?

  2. I was not able to see the output of Travis. What's wrong there?

@gasche

This comment has been minimized.

Copy link
Member

gasche commented Dec 15, 2017

The log that I have shown is from Travis. I can reproduce it on my machine (cd testsuite; make all). The error comes from the fact that

awk -v retries=1 -v max_retries=5  -f makefiles/summarize.awk <_log >_retries

returns the error code 3.

@shindere

This comment has been minimized.

Copy link
Contributor Author

shindere commented Dec 15, 2017

@shindere

This comment has been minimized.

Copy link
Contributor Author

shindere commented Dec 15, 2017

This PR actually breaks an assumption which was important.

Currently, in trunk, the exec-one target of testsuite/Makefile only takes
care of the legacy tests and that was a design choice I forgot, sorry about
that.

The motivation was precisely to be able to run the legacy and new tests
separately.

Can we perhaps close this PR and continue to live wiht the current situation
during the transition stage?

@gasche

This comment has been minimized.

Copy link
Member

gasche commented Dec 15, 2017

Closing as requested.

There should be a way to both (1) let us run the test in a give directory reliably, no matter how the tests are written and (2) give people the ability to run only legacy tests or only new test. Both features are things people want. To be investigated later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.