Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

integration tests can't test runner failures #66

Closed
ckerr opened this issue Jun 17, 2021 · 0 comments · Fixed by #108
Closed

integration tests can't test runner failures #66

ckerr opened this issue Jun 17, 2021 · 0 comments · Fixed by #108
Assignees
@ckerr
Labels
🧪 tests Improvements or additions to tests
Projects
Sprints

Comments

@ckerr
Copy link
Member

ckerr commented Jun 17, 2021

right now the electron-fiddle fake that we have in integration-tests/tests/fixtures/electron-fiddle does a good job of parroting back the right log messages for a given session, but it doesn't parrot the exit code. That's OK in our current integration tests because we're only testing a success path, but it won't work as soon as we test a failure path 🙂
@ckerr ckerr added the 🧪 tests Improvements or additions to tests label Jun 17, 2021
@ckerr ckerr added this to Unsorted in Sprints Jul 6, 2021
@ckerr ckerr moved this from Unsorted to Nice-to-Have for this Sprint in Sprints Jul 6, 2021
ckerr added a commit that referenced this issue Jul 6, 2021
@ckerr
feat: have runner bisect w/obsolete, beta, nightly
2ab1f7f 
When running `electron-fiddle bisect`, include arguments to pull in all
of the Electron releases so that the bisection range will be as small as
possible.

Fixes #66 as a side-effect.
@ckerr ckerr mentioned this issue Jul 6, 2021
Merged
@ckerr ckerr moved this from Nice-to-Have for this Sprint to In progress in Sprints Jul 6, 2021
@ckerr ckerr self-assigned this Jul 8, 2021
@ckerr ckerr closed this as completed in #108 Jul 8, 2021
ckerr added a commit that referenced this issue Jul 8, 2021
@ckerr
feat: have runner bisect w/obsolete, beta, nightly (#108)
4de3975 
* feat: have runner bisect w/obsolete, beta, nightly

When running `electron-fiddle bisect`, include arguments to pull in all
of the Electron releases so that the bisection range will be as small as
possible.

Fixes #66 as a side-effect.

* chore: comment copyediting

* refactor: use electron-fiddle --full
@ckerr ckerr moved this from In progress to Done for July 22 Sprint in Sprints Jul 10, 2021
ckerr added a commit that referenced this issue Jul 13, 2021
@ckerr
feat: run Test tasks in the runner (#115)
a43b597 
* feat: have runner bisect w/obsolete, beta, nightly

When running `electron-fiddle bisect`, include arguments to pull in all
of the Electron releases so that the bisection range will be as small as
possible.

Fixes #66 as a side-effect.

* feat: support + validate new test tasks in broker

Replaces some very bad broker validation code (of mine) with some new ow
predicates. This includes a TestJob predicate so the broker now supports
both Test and Bisect tasks.

Fixes #110.

* test: add test calling getJob() for an unknown job

* refactor: add JobType enum

* feat: support running individual tests in Runner.

Fixes #104.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ckerr ckerr

Labels
🧪 tests Improvements or additions to tests
Projects
Sprints
Done for July 22, 2021 Sprint
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: have runner bisect w/obsolete, beta, nightly electron/bugbot
1 participant
@ckerr