Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Random Ghost-Admin Travis failures #8805

Closed
kevinansfield opened this issue Aug 1, 2017 · 5 comments
Closed

Random Ghost-Admin Travis failures #8805

kevinansfield opened this issue Aug 1, 2017 · 5 comments
Assignees
Labels
affects:admin Anything relating to Ghost Admin help wanted [triage] Ideal issues for contributors to help with

Comments

@kevinansfield
Copy link
Contributor

Issue Summary

The switch to using vertical-collection in the Tags screen has resulted in a lot more random failures in Travis than usual. Many times the tests will fail with the following error:

not ok 1039 Firefox 54.0 - Acceptance: Settings - Tags when logged in loads tag via slug when accessed directly
    ---
        message: >
            tag list count after direct load: expected 0 to equal 2
        stack: >
            AssertionError@http://localhost:7357/assets/test-support-6dc08e045d81c5f7acd87d484acadd55.js:21502:18
            [3]</module.exports/Assertion.prototype.assert@http://localhost:7357/assets/test-support-6dc08e045d81c5f7acd87d484acadd55.js:16155:13
            assertEqual@http://localhost:7357/assets/test-support-6dc08e045d81c5f7acd87d484acadd55.js:16725:7
            [10]</module.exports/ctx[name]@http://localhost:7357/assets/test-support-6dc08e045d81c5f7acd87d484acadd55.js:20141:18
            _callee4$@http://localhost:7357/assets/tests-0953d936855fea30820dd1605a3ebf8f.js:4820:34
            tryCatch@http://localhost:7357/assets/vendor-8c5651d938520676ebf73bd9d810a8d6.js:12140:37
            invoke@http://localhost:7357/assets/vendor-8c5651d938520676ebf73bd9d810a8d6.js:12431:22
            defineIteratorMethods/</prototype[method]@http://localhost:7357/assets/vendor-8c5651d938520676ebf73bd9d810a8d6.js:12192:16
            step@http://localhost:7357/assets/tests-0953d936855fea30820dd1605a3ebf8f.js:4519:36
            step/<@http://localhost:7357/assets/tests-0953d936855fea30820dd1605a3ebf8f.js:4530:29
            run@http://localhost:7357/assets/vendor-8c5651d938520676ebf73bd9d810a8d6.js:9900:22
            notify/<@http://localhost:7357/assets/vendor-8c5651d938520676ebf73bd9d810a8d6.js:9913:28
            flush@http://localhost:7357/assets/vendor-8c5651d938520676ebf73bd9d810a8d6.js:7198:9

I've seen this occur in both Chrome and Firefox and it happens much less frequently when testing locally (if you can get it to appear at all there).

@kevinansfield kevinansfield added affects:admin Anything relating to Ghost Admin help wanted [triage] Ideal issues for contributors to help with tests labels Aug 1, 2017
@kirrg001
Copy link
Contributor

kirrg001 commented Dec 5, 2017

From #9301:

Ghost-Admin tests on Travis have started failing very often and it's becoming increasingly frustrating trying to get the build to pass due to failures unrelated to the build's changes. This issue is for collecting a list of the tests that randomly fail so that we can dig into why they fail and find ways to increase their reliability.

```
not ok 1593 Firefox 57.0 - Integration: Component: gh-psm-tags-input exposes all tags as options
---
    message: >
        expected 0 to equal 4
```

@kevinansfield
Copy link
Contributor Author

Another one that pops up pretty frequently:

not ok 1087 Firefox 57.0 - Acceptance: Editor when logged in renders the editor correctly, PSM Publish Date and Save Button
    ---
        message: >
            notification countdown: expected '' to include 'Post will be published in'

@kevinansfield
Copy link
Contributor Author

A new one, haven't seen this before but logging as it definitely shouldn't have failed:

not ok 1598 Firefox 57.0 - Integration: Component: gh-psm-tags-input updateTags modifies post.tags
    ---
        message: >
            You called "selectChoose('.ember-power-select-trigger', 'Tag One')" but "Tag One" didn't match any option

@kevinansfield
Copy link
Contributor Author

It's possible that these issues (or at least the power-select based tags input ones) have something to do with the browser environment. I can only get the tests to pass locally if the tests tab has focus, if the tests run whilst a different tab has focus then the same test failures are seen as reported above.

kevinansfield added a commit to kevinansfield/Ghost-Admin that referenced this issue Jan 3, 2018
refs TryGhost/Ghost#8805
- adds a test waiter for the raf scheduler which should help with the random test failures
kevinansfield added a commit to kevinansfield/Ghost-Admin that referenced this issue Jan 3, 2018
refs TryGhost/Ghost#8805
- `{{vertical-collection}}` causes async issues and random failures in our tests
- we can't upgrade to the latest version which includes a test wait helper because it's buggy in Firefox
- skip the tests for now
kevinansfield added a commit to TryGhost/Admin that referenced this issue Jan 3, 2018
…es (#938)

refs TryGhost/Ghost#8805
- `{{vertical-collection}}` causes async issues and random failures in our tests
- we can't upgrade to the latest version which includes a test wait helper because it's buggy in Firefox
- skip the tests for now
@kevinansfield
Copy link
Contributor Author

Closing in favour of a new issue with more relevant details as the tests and ember versions have now changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects:admin Anything relating to Ghost Admin help wanted [triage] Ideal issues for contributors to help with
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants