Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unit tests are not deterministic #3170

Closed
jvican opened this issue May 3, 2017 · 2 comments
Closed

Unit tests are not deterministic #3170

jvican opened this issue May 3, 2017 · 2 comments

Comments

@jvican
Copy link
Member

jvican commented May 3, 2017

Unit tests are not deterministic. There are race conditions that make some test cases fail from time to time. For instance, running ;safeUnitTests;otherUnitTests produced the following spurious error locally:

[info] SessionSettings  should
[info] ! Be identical for empty map 
[error]    MessageOnlyException: : [/data/rw/code/scala/sbt/main/target/scala-2.12/test-classes/session-settings/2.sbt.txt]:12: Pattern matching in val statements is not supported  (file:1)

We should find a solution to this.
@jvican
Copy link
Member Author

jvican commented May 3, 2017

An example in our CI: https://travis-ci.org/sbt/sbt/jobs/228286717#L2000.

@jvican jvican mentioned this issue May 10, 2017
Merged
@eed3si9n
Copy link
Member

eed3si9n commented Jan 4, 2018

@cunei found that parsing was not thread safe (#3743) since apparently thread-safety can be opted in. We tried to opt-in in #3776, but we still saw errors, so went back to synchronized in #3802.

I'm hoping that was the main cause of non-deterministic failures, so I am going to close this for now.

@eed3si9n eed3si9n closed this as completed Jan 4, 2018
cunei pushed a commit to cunei/sbt that referenced this issue May 28, 2018
Fix for sbt#4148 (SessionSettingsSpec intermittently fails)
f2a7e1f 
It turns out that `syntaxAnalyzer.UnitParser()` in global now also
needs to be synchronized. The alternative is adding `synchronizeNames = true`
in the global constructor, but that already proved unreliable in the
case of sbt#3743 (see comment sbt#3170 (comment))
@cunei cunei mentioned this issue May 28, 2018
Merged
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@eed3si9n @jvican