-
Notifications
You must be signed in to change notification settings - Fork 100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix ReplSpec tests in GH action #1216
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's try.
@@ -168,7 +168,7 @@ jobs: | |||
run: cabal build | |||
- name: Test | |||
shell: bash | |||
run: cabal test +RTS -N | |||
run: cabal run tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
alternatively, since we have some time to look at this deeply, perhaps we should take a look at https://hackage.haskell.org/package/hspec-core-2.11.0.1/docs/Test-Hspec-Core-Spec.html#v:sequential and https://hackage.haskell.org/package/hspec-core-2.11.0.1/docs/Test-Hspec-Core-Spec.html#v:parallel when running these tests in the actual PactTests.hs
file, rather than relying on CI to do "the right thing". I'll take a stab at this today and if nothing turns up, let's merge as is, but ideally we'd have these be reproducible at the local level via cabal test
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How did that analysis turn out, Emily?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hard to say - i can't get anything to trigger locally. I'm tempted to just say "merge this" and then we can re-evaluate later. I don't want to block.
@rsoeldner Let's also make note of the time it takes to complete the tests on GitHub, making sure that running them serially doesn't slow CI down too much. edit: |
@emilypi , It looks like this branch passes CI, We run tests against master and the branch of interest, for each PR. This PR runs the failing tests against master https://github.com/kadena-io/pact/actions/runs/4863641863 (If you click at the action report, checkout repository step, you see master being checked out) and then for my PR https://github.com/kadena-io/pact/actions/runs/4863600190. |
Previously, we noticed failing tests of the
ReplSpec
which rely on a pseudo-terminal. The major reason is the parallel execution of the test-suite (usingcabal tests
).This PR (should) fix the behaviour. Test PR #1215