Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Try the recommended browserstack timeouts #3468

Closed
johnjbarton opened this issue Apr 9, 2020 · 9 comments · Fixed by #3475 or #3484
Closed

Try the recommended browserstack timeouts #3468

johnjbarton opened this issue Apr 9, 2020 · 9 comments · Fixed by #3475 or #3484
Labels
released

Comments

@johnjbarton
Copy link
Contributor

karma-runner/karma-browserstack-launcher#159
@johnjbarton johnjbarton mentioned this issue Apr 16, 2020
Merged
@devoto13
Copy link
Collaborator

devoto13 commented Apr 20, 2020 •
edited

It times out after 50 seconds, which is a lot of time. I wonder if the issue is related to the fact that tests are run in 3 jobs at the same time and it can be fixed by only running them in one job.

Do you have the same error if you run tests from local machine using same BS access key as in CI?

@johnjbarton
Copy link
Contributor Author

I don't have a lot of experience with this setup. I'm inclined to try these values and see what bad happens (since we fail >50% now)

@devoto13
Copy link
Collaborator

Probably nothing bad, just not sure that it will address the issue. But let's see.

@johnjbarton johnjbarton reopened this Apr 20, 2020
@johnjbarton
Copy link
Contributor Author

just took longer to fail.

@devoto13
Copy link
Collaborator

@johnjbarton I've pushed what I had in mind into https://github.com/karma-runner/karma/tree/travis. Can you let this branch access BrowserStack variables and trigger couple of builds on it manually to see if this actually resolves the problem? If it is, I'll submit the changes as a proper PR.

@johnjbarton
Copy link
Contributor Author

johnjbarton commented Apr 20, 2020 via email

@devoto13
Copy link
Collaborator

devoto13 commented Apr 21, 2020 •
edited

Thanks! I've triggered 5 builds and all succeeded (UPD second latest build is my take on another approach, ignore it). So the solution with running these tests only in one job works.

What I suspect is causing the conflict is that all three jobs use the same build value populated from TRAVIS_BUILD_ID environment variable. I'll investigate today/tomorrow whether passing TRAVIS_JOB_ID instead would mitigate the issue, which should be cleaner solution than what is done in travis branch.

@devoto13
Copy link
Collaborator

No, the assumption is incorrect. Using TRAVIS_JOB_ID does not solve it (https://travis-ci.org/github/karma-runner/karma/jobs/677638697). Probably it is something else.

I would suggest we go with the initial approach, at least as a temporary solution. First step would be to merge #3481 and then I'll submit a PR with the actual fix.

devoto13 added a commit to devoto13/karma that referenced this issue Apr 21, 2020
@devoto13
chore(ci): fix BrowserStack timeouts
2e9dcaa 
Assuming that the issue is concurrent jobs attempting to interact with BrowserStack using the same build ID, this solution attempts to solve it by only interacting with BrowserStack in one of the jobs.

Updated build definition:

- runs complete build on the latest Node
- runs simplified build (without commit validation, lint, test:client and test:integration) on earlier versions on Node

It is not very valuable to run omitted test suits multiple times as they supposed to catch issues, which are usually Node-independent: client code fails in some browsers and API breaking changes, which will affect popular Karma plugins.

Fixes karma-runner#3468
This was referenced Apr 21, 2020
Closed
Merged
johnjbarton pushed a commit that referenced this issue Apr 21, 2020
@devoto13
chore(ci): fix BrowserStack timeouts (#3484)
3b854ad 
Assuming that the issue is concurrent jobs attempting to interact with BrowserStack using the same build ID, this solution attempts to solve it by only interacting with BrowserStack in one of the jobs.

Updated build definition:

- runs complete build on the latest Node
- runs simplified build (without commit validation, lint, test:client and test:integration) on earlier versions on Node

It is not very valuable to run omitted test suits multiple times as they supposed to catch issues, which are usually Node-independent: client code fails in some browsers and API breaking changes, which will affect popular Karma plugins.

Fixes #3468
@karmarunnerbot
Copy link
Member

🎉 This issue has been resolved in version 5.0.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

anthony-redFox pushed a commit to anthony-redFox/karma that referenced this issue May 16, 2023
@devoto13 @anthony-redFox
chore(ci): fix BrowserStack timeouts (karma-runner#3484)
480552a 
Assuming that the issue is concurrent jobs attempting to interact with BrowserStack using the same build ID, this solution attempts to solve it by only interacting with BrowserStack in one of the jobs.

Updated build definition:

- runs complete build on the latest Node
- runs simplified build (without commit validation, lint, test:client and test:integration) on earlier versions on Node

It is not very valuable to run omitted test suits multiple times as they supposed to catch issues, which are usually Node-independent: client code fails in some browsers and API breaking changes, which will affect popular Karma plugins.

Fixes karma-runner#3468
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
released
Projects
None yet
Milestone
No milestone
3 participants
@johnjbarton @devoto13 @karmarunnerbot