Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running the testsuite several times fails #7792

Closed
vicuna opened this Issue May 5, 2018 · 4 comments

Comments

Projects
None yet
2 participants
@vicuna
Copy link
Collaborator

vicuna commented May 5, 2018

Original bug ID: 7792
Reporter: @gasche
Assigned to: @gasche
Status: resolved (set by @gasche on 2018-05-30T13:15:46Z)
Resolution: fixed
Priority: low
Severity: minor
Version: 4.07.0+dev/beta2/rc1/rc2
Fixed in version: 4.08.0+dev/beta1/beta2
Category: configure and build/install

Bug description

Running the testsuite several times results in spurious failures, due to the way some tests write to their log outputs -- they append to the log of the previous run instead of replacing them.

A workaround exists, namely running "make clean" in-between runs, but this behavior still results in confusing behaviors and some wasted time.

Additional information

I have privately reported the issue to Sébastien Hinderer, the ocamltest author and maintainer, so I believe that the issue will eventually be fixed -- but it seems to be non-trivial. I am opening this issue for him to have a concrete MPR to refer to if useful, and in case other contributors would run into the same behavior and wonder if the issue is known.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented May 5, 2018

Comment author: @gasche

For the record, below is the list of spurious failures that currently occur on subsequent runs:

List of failed tests:
tests/no-alias-deps/'aliases.ml' with 1.1.1.1.1.1 (check-program-output)
tests/tool-ocamlobjinfo/'question.ml' with 1.1.1.1.1.1.1 (check-program-output)
tests/tool-debugger/basic/'debuggee.ml' with 1.1.1.1.1.1 (check-program-output)
tests/tool-debugger/no_debug_event/'noev.ml' with 1.1.1.1.1.1.1.1.1 (check-program-output)

List of unexpected errors:
tests/tool-debugger/find-artifacts/'debuggee.ml' with 1.1.1 (script)

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented May 30, 2018

Comment author: @gasche

It appears that the issue is now fixed -- it was fixed by Sébastien at some point.

@vicuna vicuna closed this May 30, 2018

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented May 31, 2018

Comment author: @gasche

Note: as of today, the issue still affects the 4.07 release and maintenance branch.

@vicuna

This comment has been minimized.

Copy link
Collaborator Author

vicuna commented Jun 1, 2018

Comment author: @gasche

(This is now fixed in 4.07 as well.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.